Simple Definition
What canon is, essentially, is a descriptor for describing whether an event, arc, or some other part of a story actually happened in that story. When we're talking about anime/manga, we often get different versions of the same story due to it being present in three main mediums: anime, manga, and light novels (other forms can exist, such as live-action adaptations and movies). Discrepancies/differences can exist between these different versions. The anime might add new story arcs, as in Naruto. So one might ask, "Is this arc canon?". And people will probably point you to the manga as the most authoritative for Naruto, since this is the source material, and will inform you that because these events and characters are not in the manga, they are non-canonical. Therefore, none of that actually happened.
We see this pattern often: the original work (usually the manga or light novels, but it can be the anime as well, as in the case of various Gundam series) is considered to be the most authoritative. The most canonical. Or in simpler words, the truest and most accurate version of events.
Not So Simple (With Examples)
In reality, canon isn't nearly so clear cut. You can, for instance, talk about an arc being canonical to the anime but not the manga. All of the filler arcs in Naruto are officially licensed episodes whether or not Kishimoto had a hand in making them. And actually, the original mangaka is sometimes involved in filler. Toriyama is alleged to have helped with filler episodes in the Dragon Ball animes, including with the design of Gregory, a filler character on King Kai's planet.
It isn't even the case that anime exclusive events are always non-canonical to the manga. For instance, in A Certain Scientific Railgun, some anime exclusive characters get brief cameos in the manga, which suggests that the arc in the anime (or some unknown variation of the arc) is actually canonical to the manga as well. Read my question and answer here for more on that.
And since we've gotten into A Certain Scientific Railgun, it should be noted that this series is itself a spin-off manga derived from the light novel series A Certain Magical Index. With Index the light novels are the source material and most authoritative, but there is no Railgun light novel series, so it is the manga that is most authoritative, and is on the same level of canon as the Index light novels. There are various other manga series as well (A Certain Scientific Accelerator, A Scientific Railgun: Astral Buddy, etc.). This collection of works is all canonical to what is referred to as the Raildex Universe.
Let us also consider the example of Fullmetal Alchemist. This is a manga series that spawned two anime series: Fullmetal Alchemist (2003) and Fullmetal Alchemist: Brotherhood. The latter follows the original manga more closely, while the former diverges in the middle and gets a different story and ending than the manga. One could say that the original anime is simply non-canonical, but I would suggest that it is more useful (and makes more sense) to say the 2003 anime follows a different canon. One could then say things such as, "This character is only canonical to the 2003 series", or "This event is shared in both canons". The manga and Brotherhood anime can certainly be said to be more authoritative, since they follow the story the mangaka created, but one needs to be flexible with this terminology so as to not make it impossible to discuss the canon of the 2003 series.
Getting back into spin-offs again, let's look at the series Isekai Quartet. This is a comedic take on several Isekai series (namely, KonoSuba, Re:Zero, Overlord, and The Saga of Tanya the Evil). While I haven't actually looked up whether the license holders have put out any statements, we can guess that Isekai Quartet is non-canonical to all four series. The absurd events therein are merely for the sake of comedy, and the tone clashes with the events in the main stories of each.
Such crossovers are often non-canonical (or at least of very dubious canon status). I recall, for instance, in the Star Wars crossover episode of Phineas and Ferb, Phineas does a meta-commentary and says something like, "Don't worry, it's all non-canon, so just have fun!"
Looking at a similar example, Nyoron! Churuya-san is a spin-off manga/anime series of the Haruhi Suzumiya franchise. It is a series of short comedic sketches with chibi style characters (as in Isekai Quartet. This type of work can often be easily labeled as non-canon to the main work. Indeed, the characters in Nyoron! Churuya-san act out of character, and other elements are quite absurd and strange and don't really fit into the framework of the main story, but are rather played up for comedy.
Not all such spin-offs/sidestories can necessarily be offhandedly dismissed as non-canonical, however. If the events can fit into the main story, sometimes all you can say is "maybe it's canonical" or "maybe it's not", using your best judgement. Chibi character designs, a heavy focus on comedy and/or parody, and meta-humor are often giveaways to the events being non-canonical (but again, this isn't true for all cases). For example, the events of Love, Chunibyo & Other Delusions! Lite are usually not so over-the-top that they couldn't exist in the actual canon. The self-referential parts of the shorts for the Chunibyo movies, however (the characters tell the audience to take a picture), certainly does lead one to suspect these particular shorts are non-canon. (Sidenote: these shorts aren't chibi-styled, but I was struggling to find a better example).
Headcanon
Which brings us to another important term: headcanon. From https://www.merriam-webster.com/words-at-play/words-were-watching-headcanon-fanon:
Headcanon generally refers to ideas held by fans of series that are not explicitly supported by sanctioned text or other media. Fans maintain the ideas in their heads, outside of the accepted canon.
Headcanon is the canon you personally accept for a story, and has nothing to do with the license holders. To use an extreme example: if you really liked a fanfiction you read, you might decide that's part of your headcanon. This type of canon can be useful for your personal enjoyment of a series, in order to fill gaps or ambiguities in the story (for example, if the story ends without love triangles being resolved), but should not be used when debating aspects of the story with others. Since everyone will likely have their own headcanon, the actual canon (as decided by the license holders) is useful for having common ground to discuss a story with other people.
Word of God
Another phrase one should be aware of with regards to canon is "Word of God". Urban Dictionary defines it as follows:
A statement regarding some ambiguous or undefined aspect of a work, the Word of God comes from someone considered to be the ultimate authority, such as the creator, director, writer or producer of a TV show / video game / film / etc.
In other words, Word of God is usually when the author of a manga makes statements about their work outside of the story. It doesn't necessarily have to be outside of the manga itself. Oda's SBS corners in One Piece come to mind.
As some other answers state, the canon is often determined as what the license holder decides is canon. In other words, Word of God decides what is canon.
I personally don't subscribe to the notion of Word of God as being authoritative, instead preferring the text of a work to be the primary authority, superseding random statements by the author. Most people do subscribe to the Word of God, however, which is why when I write answers on this site, for instance, I try to find such statements and use them when available.
Wrap Up
Others define canon as what the license holders decide is part of the story. That's correct, but only really a useful definition if the license holders are actually putting out statements on what is canonical (and to me, this definition belies how the term canonical is typically used in conversation). Granted, there are some franchises for which license holders have historically been very particular and diligent (Star Wars comes to mind, which currently has different canon categorizations for works. See also my question here.). This definition also seems insufficient for an answer to this question, particularly since what is wanted is an easy to understand explanation of canon, with particular attention to how it relates to anime. This is why I have given particular attention to topics such as filler episodes and how stories may vary across the different mediums (anime, manga, light novel), and provided several examples.
Addendum (More Examples)
With the case of Love, Chunibyo & Other Delusions!, there are some further interesting points with regard to canon. Namely, that several of the characters are anime exclusive (see this question). These characters go beyond mere filler characters: the story of the light novels has been rewritten to a very large degree to incorporate these characters. These characters form such an important part of the anime, I'd say it doesn't really make sense to refer to events of the anime as being non-canonical; rather, the anime and light novels each have their own canon.
Perhaps a better example of comedic skits being unable to offhandedly be dismissed as non-canon are the 4-Koma comics that occasionally appear in A Certain Scientific Railgun. There is, for instance, one in volume 7 where Kuroko has a plush doll and body pillow of her Oneesama. Though the tone is undoubtedly comical, Uiharu and Kuroko don't act terribly out of character here, and it's possible that some version of this conversation could actually be canonical. It would likely depend on whether or not Kuroko has some means of obtaining products with Mikoto Misaka's face on them. My favorite 4-Koma in volume 7 would have to be A Certain Misaka's No Work. This one actually tacks additional lines onto a conversation in the manga proper. One could easily imagine Misaka 10032's proclamation of "Working is for suckers" being canonical, but simply left out of the main story so as to not destroy the tone of the scene. For such an instance of an event being omitted from a scene for the sake of tone, see the last two pictures on my answer here for a Fruits Basket question. And regardless of whether Misaka's statements in this 4-Koma are literally canon, they are definitely part of my headcanon.
Further Reading
If interested in learning more, I recommend perusing the Wikipedia article and this Dragon Ball Wiki article.