Comments on: Sphinx Search in Japanese http://crunchytoast.com/2009/04/14/sphinx-search-in-japanese/ What's better than toast? Crunchytoast! Wed, 25 Jan 2012 02:55:23 +0000 hourly 1 http://wordpress.org/?v=3.3.1 By: 中国語での全文検索について調べてみた。 « fmob中の人ブログ http://crunchytoast.com/2009/04/14/sphinx-search-in-japanese/#comment-18 中国語での全文検索について調べてみた。 « fmob中の人ブログ Wed, 23 Feb 2011 06:02:33 +0000 http://crunchytoast.com/?p=131#comment-18 [...] で、現時点では日本語に対応しているかなんか微妙な感じっぽくて、いくつか日本語化のための指南がありました。 http://blog.shibu.jp/article/32831225.html http://crunchytoast.com/2009/04/14/sphinx-search-in-japanese/ [...] [...] で、現時点では日本語に対応しているかなんか微妙な感じっぽくて、いくつか日本語化のための指南がありました。 http://blog.shibu.jp/article/32831225.html http://crunchytoast.com/2009/04/14/sphinx-search-in-japanese/ [...]

]]> By: Gitorious入れたメモ « **deadwinter** http://crunchytoast.com/2009/04/14/sphinx-search-in-japanese/#comment-17 Gitorious入れたメモ « **deadwinter** Wed, 06 Jan 2010 13:36:16 +0000 http://crunchytoast.com/?p=131#comment-17 [...] http://crunchytoast.com/2009/04/14/sphinx-search-in-japanese/ [...] [...] http://crunchytoast.com/2009/04/14/sphinx-search-in-japanese/ [...]

]]> By: admin http://crunchytoast.com/2009/04/14/sphinx-search-in-japanese/#comment-16 admin Fri, 02 Oct 2009 03:59:03 +0000 http://crunchytoast.com/?p=131#comment-16 @Pat - Sorry about that! It’s all fixed. It was just one of those things. Would it be any better if I said I’ve been going around talking about Pat Allan the clothing designer??! @Pat – Sorry about that! It’s all fixed. It was just one of those things.
Would it be any better if I said I’ve been going around talking about Pat Allan the clothing designer??!

]]> By: admin http://crunchytoast.com/2009/04/14/sphinx-search-in-japanese/#comment-15 admin Wed, 30 Sep 2009 00:02:14 +0000 http://crunchytoast.com/?p=131#comment-15 @Dave - Thanks for bringing this to my attention. I've included the middle dot because I wanted it to be searchable. The middle dot is often used for separating foreign words to match their source language equivalent word separation (esp. for foreign names). More importantly, the boubiki (long vowel sound marker) character U+30FC is also in this range. See http://www.unicode.org/charts/PDF/Unicode-3.2/U32-30A0.pdf The middle dot may not be everyone's cup of tea, but the boubiki (pronounced like "bore-bikkie") is definitely needed. I may change my mind about the middle separator dot, because it will probably hinder more searches than it helps, but the "boubiki" is essential. @Dave – Thanks for bringing this to my attention. I’ve included the middle dot because I wanted it to be searchable. The middle dot is often used for separating foreign words to match their source language equivalent word separation (esp. for foreign names).

More importantly, the boubiki (long vowel sound marker) character U+30FC is also in this range. See http://www.unicode.org/charts/PDF/Unicode-3.2/U32-30A0.pdf

The middle dot may not be everyone’s cup of tea, but the boubiki (pronounced like “bore-bikkie”) is definitely needed.

I may change my mind about the middle separator dot, because it will probably hinder more searches than it helps, but the “boubiki” is essential.

]]> By: pat http://crunchytoast.com/2009/04/14/sphinx-search-in-japanese/#comment-14 pat Tue, 29 Sep 2009 22:09:31 +0000 http://crunchytoast.com/?p=131#comment-14 Hey there, great to see someone documenting how they use Sphinx and Thinking Sphinx with Japanese characters. One thing to note, though: my name's Pat Allan, not Paul Smith :) Hey there, great to see someone documenting how they use Sphinx and Thinking Sphinx with Japanese characters. One thing to note, though: my name’s Pat Allan, not Paul Smith :)

]]> By: Dave http://crunchytoast.com/2009/04/14/sphinx-search-in-japanese/#comment-13 Dave Mon, 21 Sep 2009 10:23:27 +0000 http://crunchytoast.com/?p=131#comment-13 Thanks for this! Just a quick question - you're including # Katakana U+30A0..U+30FF I don't know much about the Japanese language, but should U+30FB..U+30FF be included? (I'm asking because U+30FB is fullwidth katakana middle dot, and I thought this was used as a <a href="http://en.wikipedia.org/wiki/Interpunct#Japanese" rel="nofollow">word separator</a>)? Thanks for this!

Just a quick question – you’re including
# Katakana U+30A0..U+30FF

I don’t know much about the Japanese language, but should U+30FB..U+30FF be included? (I’m asking because U+30FB is fullwidth katakana middle dot, and I thought this was used as a word separator)?

]]> By: admin http://crunchytoast.com/2009/04/14/sphinx-search-in-japanese/#comment-12 admin Fri, 01 May 2009 15:55:59 +0000 http://crunchytoast.com/?p=131#comment-12 @richard - please check out my latest post about the <a href="http://crunchytoast.com/2009/05/01/japanese-sphinx-explained/" rel="nofollow">Sphinx Japanese Character Table</a>. It explains what each section does, and it confirms that hiragana does not match to katakana. Also I have separately confirmed that straight katakana searching does work! Can you privately send me a sample of your test data, and your setup file. I'll see what I can do. @richard – please check out my latest post about the Sphinx Japanese Character Table. It explains what each section does, and it confirms that hiragana does not match to katakana.

Also I have separately confirmed that straight katakana searching does work! Can you privately send me a sample of your test data, and your setup file. I’ll see what I can do.

]]> By: admin http://crunchytoast.com/2009/04/14/sphinx-search-in-japanese/#comment-11 admin Fri, 01 May 2009 07:27:45 +0000 http://crunchytoast.com/?p=131#comment-11 @richard - I'll look into this and email you separately. Please note, these mappings treat equivalent Katakana and Hiragana characters as the same character. This means searching for かな will also match カナ and ｶﾅ. So you basically get a phonetic search for any type of kana characters. @richard – I’ll look into this and email you separately. Please note, these mappings treat equivalent Katakana and Hiragana characters as the same character. This means searching for かな will also match カナ and ｶﾅ. So you basically get a phonetic search for any type of kana characters.

]]> By: Richard http://crunchytoast.com/2009/04/14/sphinx-search-in-japanese/#comment-10 Richard Fri, 01 May 2009 02:59:50 +0000 http://crunchytoast.com/?p=131#comment-10 Thanks for your post, I am currently testing sphinx to use in a large online shopping website in Japanese. I followed your character map guide added it to my sphinx.conf, reindexed. All queries of kanji based words are fine but katakana words are just returning all of the results in the DB? Did you find this? I am using 9.8.1 Thanks for your post, I am currently testing sphinx to use in a large online shopping website in Japanese. I followed your character map guide added it to my sphinx.conf, reindexed. All queries of kanji based words are fine but katakana words are just returning all of the results in the DB? Did you find this? I am using 9.8.1

]]>