Fantasy, along with all genres, is an artificial construct designed for marketing. But that doesn’t mean the there is no definition of Fantasy, it just means that the definition of Fantasy as a genre is “Things people who like ‘Fantasy’ would like.” If you like this book, you’re more likely to also like the other books here. When there’s a clear enough understanding of that relationship, that’s what makes a genre. Bookstores would happily make a genre of “books with cats on the cover” if there were enough of them and they thought that all of those books would appeal to the same readers.
So, when it comes to Fantasy, what is that understanding? What are people looking for when they’re looking for “Fantasy” stories? Because that should logically tell us the definition of Fantasy
If you think about it, the genre is a cluttered mess. We have Tolkien and Gaiman. Harry Potter and Narnia in the same genre as the Conan and Anita Blake books. A Song of Ice and Fire and Discworld and The Dresden Files and The Last Unicorn.
If you had to name one thing these books have in common, it’s magic, but I’m not sure that’s a good test. For one thing, both Horror and magical realism are typically separated out of Fantasy, and I think for good reason: those are often not stories Fantasy readers want to read. (I do think that publishers go a little overboard with keeping out magical realism for the sake of pretending they're Serious Books about Serious Things, but there are some that legitimately don't belong.) For another thing, while all fantasy settings that I can think of contain magic, it’s entirely possible for that magic to not come into play in an individual story. And we’re using “magic” as a catch all based solely on the explanation for supernatural occurrences or lack thereof: A story about someone who can shapeshift is about magic if they get the power from a talisman, or science if they get it from genetic manipulation. Is that really all it takes for it to appeal to Fantasy fans versus Science Fiction fans?
In a way it is; Fantasy and Sci Fi are usually shelved together because authors and fans often overlap, and sufficiently advanced science is often indistinguishable from stuff no one feels like explaining. But as someone who loves Fantasy and has only bothered to touch Sci Fi in the form of the Hitchhiker books or Spider Robinson’s Callahan’s stories, I think I can go a little deeper and talk about why I’m going to buy a book about a faerie invasion of New York and leave the book about an alien invasion of New York on the shelf.
I think the Fantasy genre is a combination of four concepts. A Fantasy novel has a heavy combination of these; some may have less of one, or even ignore one entirely, but they often make up for it with more of some of the others.
Mythic/Historic Symbolism: Swords and talismans, orcs and faeries, rituals and prophecies. These are the trappings of fantasy, it’s hard to sell a story as Fantasy without at least some hints of this. But at the same time, I don’t think it’s the raison d'etre. Instead, I think it both works in service of other thematic elements, which are the next things on the list, and as a signal to readers that those thematic elements are going to be there.
Exploration of Personal Ethics and Self-Actualization: A blog I was reading, which inspired this post, argued that Fantasy (later amended to Christian-Tradition Fantasy) is about rewarding virtue exclusively. I think it’s more fair to say that exploring virtue is an important theme that interests Fantasy fans. Specifically, exploring personal virtue, and using the trappings myth and history to explore personal values in a way that symbolically affect a larger group or world.
Some authors, like Lewis or Tolkien or Rowling, come down firmly on the side of a specific set of virtues. Others like to explore it from a more complex angle, like Martin or Gaiman or Pratchett. Barrie, with Peter Pan,could be argued to be talking about the virtue of leaving childish belief systems behind, while at the same time admitting that there are ways that kinda sucks. (In fact, implications of escapism on a personal level is practically a subgenre in Fantasy, from Thomas the Rhymer to Peter Pan to Ne and The Onion Girl.)
Many Fantasy stories then use “magic” in various forms, or a Great Men view of quasi-history, to make those personal struggles affect the fate of the kingdom/society/world. But I don’t think they’re intended to be, or usually are, read literally. Frodo sparing Gollum and leading to the destruction of the One Ring wasn’t actually about the fate of nations, it was about the individual virtues of mercy and empathy, writ large to be a more exciting and complex story. You don’t use Kant to understand Fantasy, you use Bettelheim and Campbell and Jung. 
This, I believe, is a major difference between Fantasy and Sci Fi stories. I think when Science Fiction explores ethics, from what I’ve seen, it tends to do so in an externally focused way-- how we should interact as a society and approach the future of our civilization or world; Fantasy focuses on a personal level -- how do we resolve internal and emotional-interpersonal conflict, and where do we find meaning in our lives.
Adventure: By this I mean the ever-popular combination of exploration and fighting things. I think it’s wrong to dismiss this as a thematic element; pulp stories have had a huge impact on the genre. This is shared with some kinds of Sci Fi and other genres, but I think that once again the symbolism and trappings of Fantasy give this a twist that separates Fantasy adventure from other genres: the ability to explore and battle things that do not and could never exist. You want a city to explore under modern London? Sure, we just don’t know about it because magic. Dragons that breathe fire? Magic! A forest that no one goes into even though that’s a lot of valuable acreage and it’s full of magic items and treasure? Man, those things are chocked full of creepy magic. A race of being you can murder with impunity? Orcs are Always Evil because magic!
So yeah, the advanced handwaving capabilities of Fantasy are a natural fit for adventure, and if someone just wants some orc-stabbing, castle-storming fun then Fantasy is a good genre for them.
Speculation: Another thematic element it shares with Sci Fi (leading the the "Speculative Fiction" umbrella term) is the “What If?” factor. Once again, magic is a good way to add or change specific elements without needing to engage every logical implication. It’s also useful for exploring our human history of myths and legends that often include supernatural elements, or inserting symbolic elements (like fantasy races) into real world history or to display real world issues.
Once again, I’m not arguing that every Fantasy story contains all of those; some specifically reject or subvert some of them while adhering strongly to others. And of course there are other genres, especially Sci Fi, that play on some or all of these themes, and Fantasy novels can incorporate themes from other genres. But I think that those are the things “Fantasy” readers are looking for in a book, in various combinations depending on the individual. If you wrote a book that touches on some combination of those things, you probably want it to be shelved in Fantasy.
And since we established that the Fantasy genre is a marketing distinction reflecting “things which people who like Fantasy would like,” that would mean those things are what the Fantasy genre is.
 Two interesting thoughts here that I don’t have time to explore:First, I suspect this is why “it was all a dream” (or variations like “...Or Was It?”) is somewhat less frowned upon in Fantasy-- of course it was all a dream; Fantasy as a genre is all a dream. “If we shadows have offended…”
Second, I also suspect this is why fantasy tabletop RPGs tend to be more popular; individual questions of ethics and self-advancement are easier to portray in a small group focused, largely improvised format than questions of societal ethics or advancement.