o –h<ã@sîddlZddlmZddlmmZddlmZmZm Z m Z ddl m Z Gdd„dej jjƒZGdd„dejƒZedkrud Zd ZerEd nd Zed d d d dZe edd ¡Ze d edd ¡Ze d edd ¡ZeeeefƒZeeƒdSdS)éN)Úpad_packed_sequenceÚpack_padded_sequenceÚ pack_sequenceÚPackedSequence)Ú PackedLSTMcs,eZdZdZd‡fdd„ Zd dd„Z‡ZS) Ú HLSTMCellz‰ A Highway LSTM Cell as proposed in Zhang et al. (2018) Highway Long Short-Term Memory RNNs for Distant Speech Recognition. Tcstt|ƒ ¡||_||_tj||||d|_tj||||d|_tj||||d|_ tj||||d|_ tj|d|||d|_ dS)N)Úbiasé) ÚsuperrÚ__init__Ú input_sizeÚ hidden_sizeÚnnÚLinearÚWiÚWfÚWoÚWgÚgate)Úselfr r r©Ú __class__©úU/var/www/html/env_mimamsha/lib/python3.10/site-packages/stanza/models/common/hlstm.pyr szHLSTMCell.__init__Nc Cs,| |¡|dur|j| d¡|jdd}||f}|dur)|j| d¡|jdd}| ||dd¡| ||dd¡| ||d¡t ||dgd¡}t |  |¡¡}t |  |¡¡}t |  |¡¡}t  |  |¡¡}t | t ||d|gd¡¡¡} | |||d||} |t  | ¡} | | fS)NrF)Ú requires_gradz[0]éz[1]Ú c_l_minus_one)Úcheck_forward_inputÚ new_zerosÚsizer Úcheck_forward_hiddenÚtorchÚcatÚFÚsigmoidrrrÚtanhrr) rÚinputrÚhxÚ rec_inputÚiÚfÚoÚgrÚcÚhrrrÚforwards$ "zHLSTMCell.forward)T)NN©Ú__name__Ú __module__Ú __qualname__Ú__doc__r r/Ú __classcell__rrrrrsrcs0eZdZdZ  d ‡fdd„ Zd d d „Z‡ZS) Ú HighwayLSTMz A Highway LSTM network, as used in the original Tensorflow version of the Dozat parser. Note that this is independent from the HLSTMCell above. rTFrNc stt|ƒ ¡||_||_||_||_||_||_i|_ ||_ |r#dnd|_ | |_ | |_ t ¡|_t ¡|_t ¡|_tj|dd|_|} t|ƒD]C} |j t| |d||d||d¡|j t | ||j ¡¡|j t | ||j ¡¡|jdjj ¡|jdjj ¡||j } qIdS)Nr rT)Úinplacer)Ú num_layersrÚ batch_firstÚdropoutÚ bidirectionalÚ rec_dropoutéÿÿÿÿ)r r6r r r r8rr9r:Ú dropout_stater;Únum_directionsÚ highway_funcÚpadrÚ ModuleListÚlstmÚhighwayrÚDropoutÚdropÚrangeÚappendrrÚdataÚzero_) rr r r8rr9r:r;r<r@rAÚin_sizeÚlrrrr <s6    ÿ ùzHighwayLSTM.__init__c Cs\|jdur dd„n|j}g}g}t|tƒst|||jd}t|jƒD]q}|dkr6t| |j¡|j |j |j ƒ}|durZ|d||j |d|j …|d||j |d|j …fnd}|j ||||ƒ\} \} } | | ¡| | ¡t| jt |j||jƒ¡||j||jƒƒ|j |j |j ƒ}q"|jr t||jdd}|t |d¡t |d¡ffS)NcSs|S©Nr)ÚxrrrÚ\sz%HighwayLSTM.forward..)r9rr)r@Ú isinstancerrr9rGr8rFrIÚ batch_sizesÚsorted_indicesÚunsorted_indicesr?rCrHr!r$rrDrArr") rr&Úseqlensr'r@ÚhsÚcsrLÚlayer_hxr.ÚhtÚctrrrr/[s  L  BzHighwayLSTM.forward)rTFrFrNFrMr0rrrrr67s þr6Ú__main__é Tr ré)r8r;é)r!Útorch.nnrÚtorch.nn.functionalÚ functionalr#Útorch.nn.utils.rnnrrrrÚ stanza.models.common.packed_lstmrÚmodulesÚrnnÚ RNNCellBaserÚModuler6r1ÚTÚbidirÚnum_dirÚrandnr&r'ÚcxÚoutputÚprintrrrrÚs$  /<  ÷