o h) @sxddlmZmZddlmZmZmZmZddlmZddl m Z ddl m Z GdddeZ dd ZGd d d eZd S) )Counter OrderedDict) BaseVocabBaseMultiVocab CharVocabCompositeVocab) VOCAB_PREFIX)PretrainedWordVocab) WordVocabc@seZdZdZddZdS)TagVocabz& A vocab for the output tag sequence. csZtfddjDttttfdddd_ddtjD_dS) Ncs g|] }|D]}|jqqS)idx).0sentwselfr R/var/www/html/env_mimamsha/lib/python3.10/site-packages/stanza/models/ner/vocab.py s z(TagVocab.build_vocab..cs|S)Nr )k)counterr r sz&TagVocab.build_vocab..T)keyreversecSsi|]\}}||qSr r )rirr r r z(TagVocab.build_vocab..) rdatarlistsortedkeys_id2unit enumerate_unit2idrr )rrr build_vocab s(zTagVocab.build_vocabN)__name__ __module__ __qualname____doc__r$r r r rr s r cCs|drtd|dttd}dd|D}t||dddd}t|jdt|dkr4td |jd|dkrAtd |S) Nlowerz4Did not expect an NER vocab with 'lower' set to Truer!cSsg|]}|gggqSr r )rxr r rrrz%convert_tag_vocab..langr)rr+r sepzBFailed to construct a new vocab of the same length as the originalzAFailed to construct a new vocab in the same order as the original)AssertionErrorlenrrr!) state_dictitemsvocabr r rconvert_tag_vocabsr2c@s eZdZddZeddZdS) MultiVocabcCsFt}t}|jD]\}}|||<t|j||<q ||d<|S)z= Also save a vocab name to class name mapping in state dict. _key2class)r_vocabsr0r/typer%)rstate key2classrvr r rr/ s zMultiVocab.state_dictcCsftjtjttjtjd}|}d|vsJd|d}|D]\}}||}|||||<q |S)N)rr r rr r4z-Cannot find class name mapping in state dict!)rload_state_dictr r2rr popr0)clsr/ class_dictnewr8rr9 classnamer r rr:*s zMultiVocab.load_state_dictN)r%r&r'r/ classmethodr:r r r rr3s r3N) collectionsrrstanza.models.common.vocabrrrrrstanza.models.common.pretrainr stanza.models.pos.vocabr r r2r3r r r rs