I found an excellent definition of the difference: In the presence of a kami, you pray. In the presence of a yōkai, you purify.

One might be tempted to say they are supernatural beings, but in Japan, the world is singular. There is no "otherworld" and no distinct higher or lower realms. All forms of life—humans, kami, and yōkai included—live together on the same land. The realm of the dead, Yomi no Kuni, like the underworld of the Latins and Greeks, was simply another place where those who dared, like Ulysses and Izanagi, could go. All beings in this world share, to some extent, the same nature.

All the definitions on Google's first page of results include the word "supernatural," making those definitions necessarily incorrect and unreliable. I found an article that seems definitive, so I am modifying my post accordingly. The label "yōkai" is of recent usage and therefore cannot indicate long-term characteristics of Japanese culture.

Previously, starting from the Heian period, they were called oni (鬼, usually translated as "demon") or bakemono (化け物, a word derived from bakeru 化ける, meaning "to change shape"). Oni are of Buddhist origin, the religion that introduced the concept of hell to Japan—a concept that, however, never truly took hold.

Bakemono 化け物, on the other hand, refers to a metamorphosis undergone due to a fault or disaster. It is therefore potentially reversible. These were the two classical categories.

Here you see bakemono, including an umbrella, wandering the night to frighten the living.

Since the label emerged alongside the explosion of yōkai in art, it's best not to take it too seriously. It encompasses monsters of every kind, many born from the imagination of artists, not the people. Nowadays, it's essentially just a generic name for a fantastical creature. The classical ones tended to be perceived as genuine threats, whereas literary creations obviously are not.

Among the classical ones, there are the kappa 河童, whose name literally means "river child."

The version pictured above is illustrious, being signed by Hokusai. Once, when defecating in a latrine over water was a fact of life, the kappa was feared because it was said to rip out the entrails of those defecating to eat them. This is not a literary invention and has unmistakably sinister connotations.

Then there are creatures that, strictly speaking, are the o-tsukai 御使い (messengers or envoys) of some important kami.

A very important one is the fox, or kitsune 狐.

Here is a scene from Kurosawa Akira's film "Dreams" titled Kitsune no Yomeiri (狐のお嫁入り, The Fox's Wedding).

Here is a different fox. I bought it at a shrine dedicated to Inari, the kami of anything that can lead to money and consequently the most common, revered, and beloved kami among the Japanese.

One of the main characteristics of the fox (kitsune) is its ability to "possess" women, who become its slaves. Usually, it is the fox, a female entity, that controls you. I recall my wife once refused to walk through a tunnel for fear that a fox might enter her head through the nape of her neck.

Another attribute of the fox in Japanese mythology is the capacity to assume the appearance of something or someone else. In the past, it was believed, for example, that a fox could transform into a woman, bewitch a man, and bear him a child.

Sugawara no Michizane, like Abe no Seimei—famous men in Japan—were indeed sons of a fox.

Kitsune no yomeiri (the fox's wedding) is an expression indicating sunshine showers. In one of his dreams, Kurosawa Akira takes the expression literally. The mother tells Jiro not to go outside because the fox is getting married. He disobeys, and his punishment will be terrible: his mother will not be able to let him back into the house.

Then there are the tengu. They exist in two types. One has an enormous nose—an obvious phallic symbol, indeed often used as such.

A less common variety has a duck-like beak.

The tsukumogami (九十九神 or 付喪神) [See Note] are a vast class of common objects that, after 99 years, develop a kami that animates them. Typical examples are utensils that, precisely after 99 years, somehow come to life. An example might be the umbrella pictured below.