o h* @sNddlZddlmZGdddejZGdddejZGdddejZdS)Ncs2eZdZdZfddZd ddZddZZS) WordDropouta0 A word dropout layer that's designed for embedded inputs (e.g., any inputs to an LSTM layer). Given a batch of embedded inputs, this layer randomly set some of them to be a replacement state. Note that this layer assumes the last dimension of the input to be the hidden dimension of a unit. cst||_dSN)super__init__dropprob)selfr __class__W/var/www/html/env_mimamsha/lib/python3.10/site-packages/stanza/models/common/dropout.pyr s  zWordDropout.__init__NcCsn|jr|jdkr |Sdd|D}d|d<tj|d|ji|jk}||d}|dur5|||}|S)NrcSg|]}|qSr r .0yr r r z'WordDropout.forward..device)trainingrsizetorchrandr masked_fillfloat)rx replacementmasksizedropmaskresr r r forward s zWordDropout.forwardcC d|jSNzp={}formatrrr r r extra_repr zWordDropout.extra_reprr__name__ __module__ __qualname____doc__rr r& __classcell__r r rr rs   rcs2eZdZdZd fdd ZddZddZZS) LockedDropouta A variant of dropout layer that consistently drops out the same parameters over time. Also known as the variational dropout. This implementation was modified from the LockedDropout implementation in the flair library (https://github.com/zalandoresearch/flair). Tct||_||_dSr)rrr batch_first)rrr0rr r r#  zLockedDropout.__init__cCs|jr|jdkr |S|js#|jd|d|dddd|j}n|j|dd|dddd|j}|d|j|}||S)NrrF) requires_grad)rrr0 new_emptyr bernoulli_div expand_as)rrmmaskr r r r (s,*zLockedDropout.forwardcCr!r"r#r%r r r r&4r'zLockedDropout.extra_repr)Tr(r r rr r.s  r.cs0eZdZdZfddZddZddZZS)SequenceUnitDropoutz A unit dropout layer that's designed for input of sequence units (e.g., word sequence, char sequence, etc.). Given a sequence of unit indices, this layer randomly set some of them to be a replacement id (usually set to be ). cr/r)rrrreplacement_id)rrr;rr r r;r1zSequenceUnitDropout.__init__cCsP|jr|jdkr |Sdd|D}tj|d|ji|jk}|||j}|S)z1 :param: x must be a LongTensor of unit indices. rcSr r r r r r r rDrz/SequenceUnitDropout.forward..r)rrrrrrrr;)rrrrrr r r r @s zSequenceUnitDropout.forwardcCsd|j|jS)Nzp={}, replacement_id={})r$rr;r%r r r r&IszSequenceUnitDropout.extra_reprr(r r rr r:7s   r:)rtorch.nnnnModulerr.r:r r r r s