o h>@s dZddlmZddlZddlZddlmZddlmZddl m Z m Z ddl m Z ddlmZddlmZdd lmZdd lmZeZd d d ddddZddddZdZddddZedZdddddddZddZdMd d!Zid"ej d#ej!d$ej"d%ej#d&ej$d'ej%d(ej&d)ej'd*ej(d+ej)d,ej*d-ej+d.ej,d/ej-d0ej.d1ej/d2ej0Z1gd3Z2e2D]Z3e4ee3re5ee3e1e36<qd4d5Z7dNd7d8Z8dNd9d:Z9d;d<Z:d=d>Z;d?d@ZdMdEdFZ?dGdHZ@dIdJZAdKdLZBdS)OzN Collects a few of the conparser utility methods which don't belong elsewhere )CounterN)optim)TEXTDocument) get_optimizer) SimpleModel)TransitionScheme)Tree)get_tqdmg-C6*?g?gMbP?g-C6 ?g_eG|>)adamwadadeltasgd adabeliefmadgradmirror_madgradg-q=gư>g:0yE>)rr r ?)rrr zstanza.constituency.trainerg?g{Gz?g{Gz?gv !>g>csFg}|D]}||}|fdd|jDqddt|D}|S)z Returns a list of list of tags for the items in doc doc can be anything which feeds into the pipeline(s) pipelines are a list of 1 or more retag pipelines if multiple pipelines are given, majority vote wins cs g|] }fdd|jDqS)csg|] }r |jn|jqS)xposupos.0xrr[/var/www/html/env_mimamsha/lib/python3.10/site-packages/stanza/models/constituency/utils.py 7)retag_tags...)wordsrsentencerrrr7s zretag_tags..cSsg|] }ddt|DqS)cSs"g|] }t|dddqSr)r most_commonrtagrrrr=s"r)ziprrrrr=s)append sentencesr%)doc pipelinesr tag_listspipelinerrr retag_tags,sr,Tc Cst|dkr|Sg}d}tt|d}tdt||D]}t||t|}|||}g} zt|D]\} } dd| D} | | q4Wnty]} z td| || d} ~ wwt| }t |||}tt ||D]>\}\} }z#t dd |Drt d || || |}|||d Wqoty} z td || | d} ~ wwqWdn1swYt|t|krtd t|t||S)z[ Retag all of the trees using the given processor Returns a list of new trees ritotalcSsg|] }t|jdjiqS)r)rchildrenlabel)rptrrrrSrzretag_trees..zUnable to process tree %dNcss|]}|duVqdS)Nrr#rrr ]szretag_trees..z&Tagged tree #{} with a None tag! {} {}r!z%Failed to properly retag tree #{}: {}z,Retagged tree counts did not match: {} vs {})lentqdmrangemin enumerateyield_preterminalsr& ValueErrorrr,r%any RuntimeErrorformat replace_tagsupdateAssertionError)treesr)r new_trees chunk_sizepbar chunk_start chunk_endchunkr'idxtreetokenser(r*tree_idxtagsnew_treerrr retag_treesAsJ      rNceluelugelu hardshrinkhardtanh leaky_relu logsigmoidprelurelurelu6rreluselusoftplus softshrinksoftsign tanhshrinktanh)GLU Hardsigmoid HardswishMishSiLUcCs|tvr t|Std|)zf Look up "nonlinearity" in a map from function name to function, build the appropriate layer. z/Chosen value of nonlinearity, "%s", not handled) NONLINEARITYr9 nonlinearityrrrbuild_nonlinearitys  rhFc Csd}|d}|r)d}|dd}|r|d}d}td}|d} t} d } td} n*|d }|d d}|r;|d }|d }|d}|d} |d} |d} |d} t||| d|f|| | || ||dd|dtd S)z Build an optimizer based on the arguments given If we are "multistage" training and epochs_trained < epochs // 2, we build an AdaDelta optimizer instead of whatever was requested The build_simple_adadelta parameter controls this gbert_weight_decayr stage1_bert_finetuneFstage1_bert_learning_rateg+?stage1_learning_rateNr bert_finetunebert_learning_ratelearning_beta2 learning_eps learning_rate learning_rholearning_momentumlearning_weight_decayruse_peftbert_finetune_layers) namemodellrbetasepsmomentum weight_decayrnriis_peftrv opt_logger)getDEFAULT_LEARNING_EPSDEFAULT_LEARNING_RHODEFAULT_WEIGHT_DECAYlowerrtlogger) argsrxbuild_simple_adadeltarnri optim_typermrorprqrrr|r}rrrbuild_optimizersH     rcCs\|rtjj|d|d|d|d|dd}|Stjj|d|d|d|d|dd}|S)z Build the scheduler for the conparser based on its args Used to use a warmup for learning rate, but that wasn't working very well Now, we just use a ReduceLROnPlateau, which does quite well maxlearning_rate_factorlearning_rate_patiencelearning_rate_cooldownstage1_learning_rate_min_lr)modefactorpatiencecooldownmin_lrlearning_rate_min_lr)r lr_schedulerReduceLROnPlateau)r optimizerfirst_optimizer schedulerrrrbuild_schedulers ((rcCs@|dvrtjj|j|dtj|jdd|dddSdS)zU Initializes the bias to a positive value, hopefully preventing dead neurons )rWrTrfrr!g?N)nninitkaiming_normal_weightuniform_bias)linearrgrrrrinitialize_linears"rcCsL|jdtddd|jdtddd|jdtd d d|jd d d dddS)z; Args specifically for the output location of data z --predict_dir.zWhere to write the predictions during --mode predict. Pred and orig files will be written - the orig file will be retagged if that is requested. Writing the orig file is useful for removing None and retagging)typedefaulthelpz--predict_fileNz!Base name for writing predictionsz--predict_formatz{:_O}z&Format to use when writing predictionsz--predict_output_gold_tagsF store_truezWOutput gold tags as part of the evaluation - useful for putting the trees through EvalB)ractionr) add_argumentstr)parserrrradd_predict_output_args srcCsNt|ddkst|ddkr#|ddr%d|dd|d<dSdSdS)Npredict_formatrViz{:})r3endswith)rrrrpostprocess_predict_output_argss.rcCs@|tjur t|S|tjurtj|ddSddt|DS)z Return a list of all open nodes in the given dataset. Depending on the parameters, may be single or compound open transitions. T) separate_rootcSsg|]}|fqSrrrrrrr"sz"get_open_nodes..)rTOP_DOWN_COMPOUNDr get_compound_constituentsIN_ORDER_COMPOUNDget_unique_constituent_labels)r@transition_schemerrrget_open_nodess   rc Cst||||}tdt|t||}ttjkr&tt||t|d}t |D]J\} \} } | | gd} t | D]\} }| | |sRt d | || || | || |} q<|| j}|re|}| |krtt d | || | |q*dS)zk Given a list of trees and their transition sequences, verify that the sequences rebuild the trees z/Verifying the transition sequences for %d treesr-rzpTree {} of {} failed: transition {}:{} was not legal in a transition sequence: Original tree: {} Transitions: {}zsTree {} of {} failed: transition sequence did not match for a tree! Original tree:{} Transitions: {} Result tree:{}N)rrinfor3r%getEffectiveLevelloggingINFOr4r7initial_state_from_gold_treesis_legalr;r<applyget_top_constituent constituentsreverse)r@ sequencesr unary_limitrrw root_labelsrxdatarKrHsequencestaterGtransresultrrrverify_transitions%s$   rc Cst|}|D]>}||vrEd}d}t|D]\}} t| }||vr,|d7}|dur,|}qd||||d||} |r@t| t| qdS)zY Check that all the constituents in the other dataset are known in the train set Nrr!a'Found constituent label {} in the {} set which don't exist in the train set. This constituent label occured in {} trees, with the first tree index at {} counting from 1 The error tree (which may have POS tags changed from the retagger and may be missing functional tags or empty nodes) is: {:P})r rr7r<r;warningswarn) train_constituentsr@ treebank_namefailrcon first_error num_errorsrKrHerrorrrrcheck_constituents=s$   rcCs,t|D]}||vrtd||qdS)zX Check that all the root states in the other dataset are known in the train set zLFound root state {} in the {} set which is not a ROOT state in the train setN)r get_root_labelsr;r<)r other_treesr root_staterrrcheck_root_labelsRs rcCsjg}t}|D]}d|}||vrq||||qt|t|kr3tdt|t|||S)z2 Filter duplicates from the given dataset z{}z&Filtered %d duplicates from %s dataset)setr<addr&r3rr)r@rrA known_treesrHtree_strrrrremove_duplicate_treesZs   rcCs>dd|D}t|t|dkrtdt|t||S)z~ remove trees which are just a root and a single word TODO: remove these trees in the conversion instead of here cSs|g|]:}t|jdks:t|jdkrt|jdjdks:t|jdkrt|jdjdkrt|jdjdjdkr|qSr )r3r/rrrrrps "<z*remove_singleton_trees..rz*Eliminated %d trees with missing structure)r3rr)r@rArrrremove_singleton_treesjsr)T)F)C__doc__ collectionsrrrtorch.nnrtorchrstanza.models.common.docrrstanza.models.common.utilsr%stanza.models.constituency.base_modelr,stanza.models.constituency.parse_transitionsr%stanza.models.constituency.parse_treer stanza.utils.get_tqdmr r4DEFAULT_LEARNING_RATESrrDEFAULT_MOMENTUM getLoggerrrr,rNCELUELUGELU HardshrinkHardtanh LeakyReLU LogSigmoidPReLUReLUReLU6RReLUSELUSoftplus SoftshrinkSoftsign TanhshrinkTanhrenonlinearity_listrghasattrgetattrrrhrrrrrrrrrrrrrrrs            L        /