o hdU@sdZddlmZmZddlZddlmZddlmZddl Z ddl Z ddl Z ddl m Z dZdZd Zd Ze d Zd ZGd ddeZGddde ZdS)z Tree datastructure )dequeCounterN)Enum)StringIO) StanzaObject) (z[-=#*])z*E*z*T*z*O*c@s$eZdZdZdZdZdZdZdZdS)TreePrintMethodzz Describes a few options for printing trees. This probably doesn't need to be used directly. See __format__ N) __name__ __module__ __qualname____doc__ONE_LINELABELED_PARENSPRETTYVLSP LATEX_TREEr r r `/var/www/html/env_mimamsha/lib/python3.10/site-packages/stanza/models/constituency/parse_tree.pyr !sr c@s4eZdZdZdBddZddZddZd d Zd d Zd dZ ddZ dCddZ ddZ ddZ ddZddZdDddZeddZedd Zed!d"Zed#d$Zed%d&ZedEd(d)Zed*d+ZedFd-d.Zefd/d0Zd1d2Zd3d4Zd5d6Zd7d8Zd9d:Z d;d<Z!d=d>Z"edGd@dAZ#dS)HTreez4 A data structure to represent a parse tree NcCs8|durt|_nt|tr|f|_nt||_||_dSN)EMPTY_CHILDRENchildren isinstancertuplelabel)selfr!rr r r__init__2s     z Tree.__init__cCst|jdkS)Nrlenrr"r r ris_leaf<sz Tree.is_leafcCs"t|jdkot|jdjdkS)Nr rr$r&r r ris_preterminal?s"zTree.is_preterminalccsx|r |VdS|rtdt|j}t|d}|dur:|r(|Vnt|j|}t|d}|dus dSdS)z? Yield the preterminals one at a time in order Nz6Attempted to iterate preterminals on non-internal node)r(r' ValueErroriterrnext itertoolschain)r"iteratornoder r ryield_preterminalsBs   zTree.yield_preterminalscCs&|r|jgSdd|D}|S)z. Get the labels of the leaves cSsg|]}|jdjqS)r)rr!.0xr r r ]z$Tree.leaf_labels..)r'r!r0)r"wordsr r r leaf_labelsVszTree.leaf_labelscCs t|Sr)r%r7r&r r r__len__`s z Tree.__len__cCs,|rdS|r dStdd|jDS)zT Returns True if all leaves are under preterminals, False otherwise FTcs|]}|VqdSr)all_leaves_are_preterminalsr2tr r r mz3Tree.all_leaves_are_preterminals..)r'r(allrr&r r rr:cs z Tree.all_leaves_are_preterminalsc Cs:|durdd}d}t}t}||t|dkr|}|turT|turD|d8}|tt|dkrzz#Tree.pretty_print..rr  z z %s%s %s%scsr9r)r(r1r r rr=r>z$Tree.pretty_print..z%s%sz %s%s %s%sz%s%s )rrappendr%pop CLOSE_PARENwriter( OPEN_PARENr!rr?reversedseekread)r" normalizeindentbufstackr/childr r r pretty_printos\       (  *     &&zTree.pretty_printc sdtj}|dkrtj}dn|r|ddkrtj}|dn|dkr'tj}n|r7|ddkr7tj}|dnt|dkr?tj}nl|rO|ddkrOtj}|dn\|ra|ddkratj}|dd k}nJ|r}t|d kr}|d dkr}tj}|d|dd k}n.|d krtj}n&|rt|d kr|d d krtj}|dn|r|dtjd d d|tjkrdnd|tjkrdndfdd}|tjur| |St o}t }|tjkr|r| d |jn| dt|jdkrtdt|jd kr tdt|j||jdnZ|tjkrh| dt|jdkr*tdt|jd krSt|jdjdkrS| d| ||jdj| dn|jdkrb||jdn ||n||t|dkr'|}t|tr| |qmt|jdkr|jdur| ||jqm|tjur|r| ||jdjqm| d||j|dnH|tjus|tjur| t|jdur| ||j|tn#|tjur | d t||jf|td||j|tt|jD] }|||tqt|dkst|tjkr2| d!|d|WdS1sFwYdS)"aT Turn the tree into a string representing the tree Note that this is not a recursive traversal Otherwise, a tree too deep might blow up the call stack There is a type specific format: O -> one line PTB format, which is the default anyway L -> open and close brackets are labeled, spaces in the tokens are replaced with _ P -> pretty print over multiple lines V -> surround lines with ..., don't print ROOT, and turn () into L/RBKT ? -> spaces in the tokens are replaced with ? for any value of ? other than OLP warning: this may be removed in the future ?{OLPV} -> specific format AND a custom space replacement Vi -> add an ID to the in the V format. Also works with ?Vi rL_rHrOPVir Tz]Use of a custom replacement without a format specifier is deprecated. Please use {}O insteadr ) stacklevelLBKTr@RBKTrAcs|dddS)Nrr rrB)textLRBRRBspace_replacementr rrQsz"Tree.__format__..normalizez z z(Cannot print an empty tree with V formatz2Cannot print a tree with %d branches with V formatz\Tree z(Cannot print an empty tree with T formatz[.? z ]ROOTNz[.%sz%s_%sz )r rrrrr%rwarningswarnformatrVrrrLtree_idrr)rIr!rJrstrr(rMrKSPACE_SEPARATORrNrOrP) r"spec print_format use_tree_idrQrSrTr/rUr rbr __format__s           &                    &zTree.__format__cCs d|S)N{})rir&r r r__repr__!s z Tree.__repr__cCsf||urdSt|ts dS|j|jkrdSt|jt|jkr!dStddt|j|jDr1dSdS)NTFcss|] \}}||kVqdSrr )r2c1c2r r rr=-szTree.__eq__..)rrr!r%ranyzip)r"otherr r r__eq__$s  z Tree.__eq__cCs"|jsdSdtdd|jDS)Nrr csr9r)depthr1r r rr=4r>zTree.depth..)rmaxr&r r rry1sz Tree.depthcCsV|r |r ||n|r|r||n|r|||jD] }||||qdS)a- Visit the tree in a preorder order Applies the given functions to each node. internal: if not None, applies this function to each non-leaf, non-preterminal node preterminal: if not None, applies this functiion to each preterminal leaf: if not None, applies this function to each leaf The functions should *not* destructively alter the trees. There is no attempt to interpret the results of calling these functions. Rather, you can use visit_preorder to collect stats on trees, etc. N)r'r(rvisit_preorder)r"internal preterminalleafrUr r rr{6s  zTree.visit_preordercCs*t|tr|g}t|}tt|S)zi Walks over all of the trees and gets all of the unique constituent names from the trees )rrget_constituent_countssortedsetkeys)trees constituentsr r rget_unique_constituent_labelsOs  z"Tree.get_unique_constituent_labelscs8t|tr|g}t|D] }|jfdddq S)zo Walks over all of the trees and gets the count of the unique constituent names from the trees c|jgSrupdater!rDrr rrEcz-Tree.get_constituent_counts..)r|)rrrr{rtreer rrrYs zTree.get_constituent_countsc<t|tr|g}t|D] }|jfdddq tS)z\ Walks over all of the trees and gets all of the unique tags from the trees c |jSraddr!rDtagsr rrEp z&Tree.get_unique_tags..)r}rrrr{rrr rrget_unique_tagsf zTree.get_unique_tagscr)z] Walks over all of the trees and gets all of the unique words from the trees crrrrDr6r rrE}rz'Tree.get_unique_words..r~rrr rrget_unique_wordssrzTree.get_unique_wordscs`|dkrtSt|tr|g}t|D] }|jfdddqtddd|DS)z[ Walks over all of the trees and gets the most frequently occurring words. rcrrrrDrr rrErz'Tree.get_common_words..rcs|]}|dVqdSrNr r1r r rr=r>z(Tree.get_common_words..N)rrrrr{r most_common)r num_wordsrr rrget_common_wordss zTree.get_common_words皙?cspt|tr|g}t|D] }|jfdddq ttt|d}tddd| ddDS) z Walks over all of the trees and gets the least frequently occurring words. threshold: choose the bottom X percent crrrrDrr rrErz%Tree.get_rare_words..rr csrrr r1r r rr=r>z&Tree.get_rare_words..NrH) rrrr{rzintr%rr)r thresholdrr rrget_rare_wordss &zTree.get_rare_wordscCsttdd|DS)Ncs|]}|jVqdSrr!r1r r rr=z'Tree.get_root_labels..)rr)rr r rget_root_labelsszTree.get_root_labelsFcCst}t}|D]r}|r||jf|jD]}||qn||t|dkrz|}|s6| r7q$|jg}t|jdkrb|jd sb|jd}||jt|jdkrb|jd rI|t ||jD]}||qlt|dks*qt |S)Nrr ) rrrr!rrIr%rJr'r(r r)r separate_rootrrTrrUr/labelsr r rget_compound_constituentss0          zTree.get_compound_constituentscsR|j}|r|st|dkr|dvr|d}fdd|jD}t||S)zn Return a copy of the tree with the -=# removed Leaves the text of the leaves alone. r )r@rArcg|]}|qSr )simplify_labelsr2rUpatternr rr4z(Tree.simplify_labels..)r!r'r%splitrr)r"r new_label new_childrenr rrrs   zTree.simplify_labelscCs2|r t|jSddt|jD}t|j|S)z Flip a tree backwards The intent is to train a parser backwards to see if the forward and backwards parsers can augment each other cSg|]}|qSr )reverserr r rr4rFz Tree.reverse..)r'rr!rNrr"rr r rrs  z Tree.reversecs\|r t|jS|rt|jt|jdjS|j|j}t|fdd|jDS) Copies the tree with some labels replaced. Labels in the map are replaced with the mapped value. Labels not in the map are unchanged. rcrr )remap_constituent_labelsr label_mapr rr4rz1Tree.remap_constituent_labels..)r'rr!r(rget)r"rrr rrrs  zTree.remap_constituent_labelscs\|r|j|j}t|S|r t|j|jdSt|jfdd|jDS)rrcrr ) remap_wordsrword_mapr rr4rz$Tree.remap_words..)r'rr!rr(rr)r"rrr rrrs zTree.remap_wordscs<t|fdd|}tddDrtd|S)zs Replace all leaf words with the words in the given list (or iterable) Returns a new tree csF|rtd}|durtdt|St|jfdd|jDS)Nz&Not enough words to replace all leavescsg|]}|qSr r r1)recursive_replace_wordsr rr4rFzGTree.replace_words..recursive_replace_words..)r'r+r)rr!r)subtreewordr word_iteratorr rrs  z3Tree.replace_words..recursive_replace_wordscs|]}dVqdSTNr r2rXr r rr=z%Tree.replace_words..z!Too many words for the given tree)r*rur))r"r6new_treer rr replace_wordss zTree.replace_wordscCs|rtdt|trdd|D}nt|}t|}t}| |t |dkre| }| rLzt |}Wn tyGtdw||_n|rWtd||t|jt |dks.tdd|Drrtd|S) Nz$Must call replace_tags with non-leafcsrrrr1r r rr= rz$Tree.replace_tags..rz*Not enough tags in sentence for given treezGot a badly structured tree: {}csrrr rr r rr=!rz Too many tags for the given tree)r'r)rrr0r*copydeepcopyrrIr%rJr(r+ StopIterationr!riextendrNrru)r"r tag_iteratorrqueue next_noder!r r r replace_tagss0        zTree.replace_tagscCs|r t|jS|r(|jdks|jdjtvrdSt|jt|jdjSdd|jD}dd|D}t|dkr?dSt|j|S)a Return a copy of the tree, eliminating all nodes which are in one of two categories: they are a preterminal -NONE-, such as appears in PTB *E* shows up in a VLSP dataset they have been pruned to 0 children by the recursive call z-NONE-rNcSrr ) prune_nonerr r rr45rFz#Tree.prune_none..cSsg|]}|dur|qSrr rr r rr46r5)r'rr!r(rWORDS_TO_PRUNEr%rr r rr's   zTree.prune_nonecCs|s|r dSt|jdkrM|}d}|s<|sz)Tree.count_unary_depth..csr9rrr;r r rr=Gr>)r(r'r%rrz)r"r<score child_scorer r rr;s  zTree.count_unary_depthrqcCsXt|ddd}|D]}||||dq WddS1s%wYdS)Nwzutf-8)encodingrG)openrLri)rout_filefmtfoutrr r rwrite_treebankJs  "zTree.write_treebank)NNr)NNN)r)F)rq)$rrrrr#r'r(r0r7r8r:rVrprrrxryr{ staticmethodrrrrrrrrCONSTITUENT_SPLITrrrrrrrrrr r r rr.sR    :x            r)r collectionsrrrenumriorr,rerg"stanza.models.common.stanza_objectrrKrlrMrcompilerrr rr r r rs"