All of the components of the bike (frame, gears, derailleur, brakes, shifters, saddle, grips, suspension, etc) are what you interact with on the bike. Higher quality components will offer better experiences overall. In addition to looking for quality components in general, I will go over a couple of specific points to consider that might not be obvious to newcomers:
Brakes: Because it is easier to go faster on an eBike, you may need more braking power than usual, so you will want to pay particular attention to the brakes. You should look for eBikes that have disc breaks.
Gears: Many eBikes don't include any gears at all, and that can be OK for many people's needs. The motor can provide the extra power on the hills that gears are designed to make easier. Not only does not having gears make the bike more affordable, but it is also is quieter, easier to maintain, and easier to use. For beginners eBikes can be more confusing than "acoustic" bikes because you have to learn how to use both the gears and the power assist levels. On the other hand, having gears AND a motor is really the best of both worlds and if you are interested in longer rides and or commuting to and from work, gears will get you more versatility and efficiency.
Shifters: You may want to consider avoiding eBikes that change gears or power assist levels/throttle by twisting the entire hand grip. It can be confusing and dangerous to accidentally affect gearing, power assist levels, or throttle by inadvertently moving your wrist. Using levers, buttons, or twist shifts next to the main grip are much safer.
Suspension: Some eBikes offer front and/or rear suspension. Suspension can really make longer rides more comfortable.
I want to give a special callout to a particularly nice option that you should consider if you can afford it, and that is a bike with a belt driven internal gear hub. There are two components at work here, the belt drive system, and the internal gear hub.
Belt Drive System: Typically bikes use a metal chain to drive the bike. Chains require routine maintenance (cleaning, lubrication, and tightening), and wear out over time. A Belt Drive System uses a belt (like motorcycles) that doesn't need cleaning or lubricating, and lasts much longer than metal chains. The belts are also significantly quieter than the metal chains, and they also don't get grease on your pants.
Internal Gear Hub: Traditionally bikes have a "cassette" which contains several sprockets of various sizes. By moving the chain from one sprocket to another using a derailleur, you affect the "gear ratio" of the bike, which can make it easier to go up hills by shifting into lower gears. The number of sprockets a bike has determines how many gears the bike has (typically six to nine on an eBike). Internal gear hubs use completely different technology and don't have sprockets or derailleurs. Internal gear hubs have a number of advantages over cassette sprocket systems: Sprockets and derailleurs need routine maintenance. The cables that control the derailleur's movement has to be perfectly calibrated to ensure the chain accurately moves to the correct sprocket and does not rub on the derailleur. The movement of the chain through the derailleur and sprockets cause noise. Derailleurs can easily be bent out of alignment because they protrude from the side of the bike. In order to change gears, you have to be pedaling. Internal gear hubs on the other hand are almost completely silent, and require virtually no maintenance, and you can change gears while pedaling or while at a stop. Additionally, instead of being limited to the number of gears (and what the exact gear ratio is) based on how many sprockets the bike has and what size they are, internal gear hubs have Continuously Variable Transmissions (CVTs), which means that you can set virtually any gear ratio between the minimum and maximum gear of the hub (and typically internal gear hubs have a wider gear range than traditional cassette sprockets).
Combing a belt drive and an internal gear hub results in some very nice benefits:
virtually maintenance free (no chain cleaning , lubing, tightening, and replacing; no derailleur adjusting)
virtually silent operation (no chain noise, no derailleur noise, no sprocket noise)
you can change gears while pedaling or while stopped
more gears to chose from and a wider gear ratio
no grease on your pants from the chain
A few bikes with a Belt Driven Internal Gear Hub: Gazelle Ultimate C380 and C380+, Priority Current, some Treks, some Giant, some Specialized