Skip to content

Python CLI ​

The Python wheel installs suzume, a command for analysis and tag extraction. It uses the same bundled native library and dictionaries as the Python API.

Installation ​

bash
pip install suzume

The command is available on the same platforms as the Python package: Linux x86_64 (manylinux2014 / manylinux_2_17) and macOS arm64 on macOS 11 or later. See Python Bindings for the full compatibility list.

Input ​

Pass text as arguments or pipe UTF-8 text to standard input:

bash
suzume "東京へ行く"
printf 'りんごを食べる\n' | suzume

When text is supplied as arguments, the CLI joins the arguments with single spaces. With no arguments, non-interactive standard input is read as one document. It removes one leading UTF-8 BOM and one trailing LF (or CRLF pair), while preserving interior line breaks.

analyze is an optional alias. These commands are equivalent:

bash
suzume --mode search "東京の公園"
suzume analyze --mode search "東京の公園"

Use -- before input that begins with a hyphen or is exactly analyze, version, or help. The CLI treats those three words as commands when they are the first operand before --. After --, every operand is text.

If there is no operand and standard input is an interactive terminal, the command prints suzume: error: no input text provided to standard error and exits with status 2. An explicit empty operand ("") or empty or newline-only standard input produces no output and exits with status 0.

Output formats ​

Select the format with -f or --format:

FormatOutput
tsvDefault. surface TAB POS TAB base_form TAB start TAB end
tagstag TAB POS, one tag per line
jsonAn object containing input, normalized_text, and morphemes (see JSON and offsets)
chasenChaSen-like fields (surface, *, base_form, Japanese POS, conjugation type, conjugation form) followed by EOS. The conjugation type and form columns hold a value only when the token has one, which only VERB, ADJ and AUX tokens can (美しい has a form but no type); they are * for every other token
bash
suzume --format json "ABCを検索"
suzume --format chasen "食べている"
suzume --format tags "東京の公園に行く"

Backslashes, tabs, and line breaks inside one-line TAB output are escaped.

Analysis options ​

OptionDescription
-m, --mode MODEUse normal, search, or split segmentation
--no-preserve-vuFold the ヴ series to the バ row (ヴァイオリン → バイオリン); the default keeps it
--no-preserve-caseConvert ASCII letters to lowercase
--preserve-symbolsKeep punctuation-like SYMBOL tokens; content symbols and emoji remain OTHER either way
--no-lemmatizeKeep source base forms instead of correcting them after analysis; POS and conjugation annotations are unchanged (details)
--merge-compoundsMerge consecutive noun compounds
--skip-env-configIgnore scorer configuration environment variables
bash
suzume --mode split --no-preserve-case "ABCと東京都"
suzume --no-preserve-vu --preserve-symbols "ヴァイオリン、ビオラ"

Currency and unit signs, arrows, technical marks, and emoji remain OTHER by default. --preserve-symbols adds punctuation-like tokens such as 、 and 。.

Long options that take a value accept both --option value and --option=value. A short option takes its value as a separate argument (-d PATH); an attached form such as -dPATH is an unknown option. A value-taking option never consumes a following argument that starts with - and is longer than one character: the command prints suzume: error: missing value for <option as typed> and exits with status 2. A bare - is a valid value. To pass a value that starts with a hyphen, use the long option as --option=value.

Run suzume --help for the current option summary. suzume --version, suzume -v, and suzume version each print one line, suzume <version>, and exit with status 0, even when other arguments follow. A usage error elsewhere on the line is reported first, with status 2.

Dictionaries ​

Use -d or --dictionary to load a UTF-8 text dictionary or compiled .dic file. The option is repeatable:

bash
suzume --dictionary terms.tsv --dictionary names.dic "東京スカイツリーへ行く"
OptionDescription
-d, --dictionary PATHLoad a text or compiled .dic dictionary; repeat to load more
--skip-core-dictionaryDo not load the bundled core dictionary
--skip-bundled-user-dictionaryDo not load the bundled user dictionary; dictionaries passed with -d still load

Warnings produced while the analyzer is being created, and after every -d load, are written to standard error as warning: <text> lines, for example warning: Skipped dictionary record at line 1: missing POS field. If a dictionary cannot be loaded, the command prints suzume: error: failed to load dictionary: <path>: <library message> and exits with status 1. The file's bytes go to the library unchanged, so a source that is not UTF-8 reports suzume: error: failed to load dictionary: <path>: Dictionary entry is not valid UTF-8 at line 1, with the line number of the first bad entry.

Tag output ​

Choose --format tags to generate keyword tags:

bash
suzume --format tags --tag-max-tags 5 "東京都の天気予報を確認する"
OptionDefaultDescription
--tag-pos-filter NAMEallKeep one POS; repeat for more
--tag-exclude-basicoffExclude tags whose base form contains only hiragana
--tag-no-use-base-formoffUse surface forms instead of base forms
--tag-min-length N2Set the minimum tag length (0 to 4294967295)
--tag-max-tags N0Limit the number of tags (0 means unlimited; 0 to 4294967295)
--tag-no-exclude-particlesoffKeep particles
--tag-no-exclude-auxiliariesoffKeep auxiliaries
--tag-no-exclude-formal-nounsoffKeep formal nouns
--tag-no-exclude-low-infooffKeep low-information words
--tag-no-remove-duplicatesoffKeep duplicate tags

--tag-pos-filter accepts NOUN, VERB, ADJ, ADV, PARTICLE, and AUX, case-sensitive; any other name is a usage error. NOUN also admits pronouns. Naming a part of speech admits it, so --tag-pos-filter PARTICLE returns particles by itself, without --tag-no-exclude-particles.

A --tag-min-length or --tag-max-tags value above 4294967295 prints suzume: error: invalid --tag-min-length value: X (or --tag-max-tags) and exits with status 2.

Any --tag-* flag together with a format other than tags is a usage error: the command prints suzume: error: --tag-* options require -f tags and exits with status 2. -f tags with no resulting tags prints nothing and exits with status 0.

bash
suzume --format tags \
  --tag-pos-filter NOUN \
  --tag-pos-filter VERB \
  --tag-min-length 1 \
  "本を読む"

JSON and offsets ​

Token start and end values are Unicode code-point offsets into the normalized text, not necessarily the original input; see Offsets. JSON output includes that exact string as normalized_text. Each entry in morphemes has these keys:

KeyValue
surfaceSurface form
posPart of speech in English
base_formBase form
pos_jaPart of speech in Japanese
conj_type, conj_formConjugation type and form for VERB, ADJ, and AUX; null when absent
extended_posExtended POS code
start, endCode-point offsets into normalized_text
is_user_dict, is_formal_noun, is_low_info, is_unknown, is_from_dictionaryBoolean flags, as in the Python Morpheme
scoreCost the analyzer assigned to the token; lower is more likely, and values are not comparable across inputs

Keys appear in the order surface, pos, base_form, pos_ja, conj_type, conj_form, extended_pos, start, end, is_user_dict, is_formal_noun, is_low_info, is_unknown, is_from_dictionary, score.

bash
suzume --no-preserve-case --format json "ABCを検索"

Use JSON when normalization can change the input and downstream code needs to slice text by the reported offsets.

Mapping to Python options ​

Each flag sets the matching Suzume() or generate_tags() option from the Python API. Flags that negate a default are written with no:

FlagPython option
-m, --modemode
--no-preserve-vupreserve_vu=False
--no-preserve-casepreserve_case=False
--preserve-symbolspreserve_symbols=True
--no-lemmatizelemmatize=False
--merge-compoundsmerge_compounds=True
--skip-core-dictionaryskip_core_dictionary=True
--skip-bundled-user-dictionaryskip_bundled_user_dictionary=True
--skip-env-configskip_env_config=True
-d, --dictionaryload_user_dictionary() for text, load_binary_dictionary() for a path ending in .dic
--tag-pos-filterpos_filter
--tag-exclude-basicexclude_basic=True
--tag-no-use-base-formuse_base_form=False
--tag-min-length, --tag-max-tagsmin_length, max_tags
--tag-no-exclude-particles, --tag-no-exclude-auxiliaries, --tag-no-exclude-formal-nouns, --tag-no-exclude-low-infothe matching exclude_*=False
--tag-no-remove-duplicatesremove_duplicates=False

Environment variables ​

The command reads these variables through the native library. --skip-env-config ignores the scorer variables.

VariableEffect
SUZUME_SCORER_CONFIGPath to a JSON file of scorer-option overrides
SUZUME_SCORER_{SECTION}_{KEY}Overrides one scorer option. SECTION is JOIN, SPLIT, UNARY, BIGRAM, VERB, or INFL; KEY is the option field name
SUZUME_LIB_PATHPath to a shared library to load instead of the one in the wheel

An installed wheel always reads its bundled dictionaries from the package directory; it never consults SUZUME_DATA_DIR or ~/.suzume, and the command has no data-directory flag.

Scope ​

The Python suzume command covers analysis and tag extraction. It does not compile or validate dictionaries and does not run the native test or benchmark commands.

Use the separately built suzume-cli for those developer workflows. See Native CLI Reference and Native Build.

See also ​