One that I know is NiconiCommons (Japanese) (sister site of NicoNicoDouga, a Japanese video sharing site). Basically it's a site where users can upload and share resources/materials (image, sound, video, etc) for people to use.
You can browse all the audio materials, or search the one you want.
Click Advanced Search button
Input the Tag or Keyword in Japanese
Tick 音声 (onsei, audio) only
Press Search button
Alternatively, replace the
[keyword] from these URL:
Tag-only : http://commons.nicovideo.jp/search/tag/[tag]?s=d&o=d&t=2&sc=1
Keyword-only : http://commons.nicovideo.jp/search/keyword/[keyword]?s=d&o=d&t=2&sc=1
Tag & Keyword: http://commons.nicovideo.jp/search/hybrid/[tag]/[keyword]?s=d&o=d&t=2&sc=1
Example: List of JoJo's Bizarre Adventure sound bites.
However, since these are all uploaded by users, you may not find what you want. You also need to have an account to be able to download, but the registration page is available in English.