o hX/ @sUdZddlmZmZmZmZmZmZddlm Z gdZ dZ iZ ee efed<GdddZGd d d eZGd d d eZGd ddeZde dedeefddZdeedededeefddZdee effddZdS)z] emoji.tokenizer ~~~~~~~~~~~~~~~ Components for detecting and tokenizing emoji in strings. )List NamedTupleDictUnionIteratorAny) unicode_codes) EmojiMatch EmojiMatchZWJEmojiMatchZWJNonRGITokentokenize filter_tokensu‍ _SEARCH_TREEc @seZdZdZdZdedededeeee fdffdd Z d eee ffd d Z d e fd dZ d edfddZd efddZdS)r zd Represents a match of a "recommended for general interchange" (RGI) emoji in a string. emojistartenddatarrrrNcCs"||_ ||_ ||_ ||_dSNr)selfrrrrrJ/var/www/html/env_mimamsha/lib/python3.10/site-packages/emoji/tokenizer.py__init__"szEmojiMatch.__init__returncCs6|jr|j}|j|d<|j|d<|S|j|jdS)z Returns a copy of the data from :data:`EMOJI_DATA` for this match with the additional keys ``match_start`` and ``match_end``. match_start match_end)rr)rcopyrr)remj_datarrr data_copy1s    zEmojiMatch.data_copycCs t|jvS)zp Checks if this is a ZWJ-emoji. :returns: True if this is a ZWJ-emoji, False otherwise )_ZWJrrrrris_zwj>s zEmojiMatch.is_zwj)r r cCs|rt|S|S)z Splits a ZWJ-emoji into its constituents. :returns: An :class:`EmojiMatchZWJ` containing the "sub-emoji" if this is a ZWJ-emoji, otherwise self )r"r r!rrrsplitGszEmojiMatch.splitcCs&|jjd|jd|jd|jdSN(z, :)) __class____name__rrrr!rrr__repr__Ss&zEmojiMatch.__repr__)r) __module__ __qualname____doc__ __slots__strintrrrrrboolr"r#r*rrrrr s     r cs^eZdZdZdZdeffdd ZdefddZde fd d Z dd d Z defd dZ Z S)r zr Represents a match of multiple emoji in a string that were joined by zero-width-joiners (ZWJ/``\u200D``).)emojismatchc szt|j|j|j|jg|_ |j}|jtD] }t |||t |t j |d}|j||t |d7}qdS)N)superrrrrrr2r#r r lenr EMOJI_DATAgetappend)rr3iemr(rrr^s  zEmojiMatchZWJ.__init__rcCstdd|jDS)z1 Joins a ZWJ-emoji into a string cs|]}|jVqdSrr.0r;rrr oz%EmojiMatchZWJ.join..)r joinr2r!rrrrDjszEmojiMatchZWJ.joincCsdS)NTrr!rrrr"qzEmojiMatchZWJ.is_zwjcCs|Srrr!rrrr#trEzEmojiMatchZWJ.splitcCs(|jjd|d|jd|jdSr$)r(r)rDrrr!rrrr*ws(zEmojiMatchZWJ.__repr__)rr )r)r+r,r-r.r rr/rDr1r"r#r* __classcell__rrr=rr Ws  r c@s8eZdZdZdedefddZddZdefd d Zd S) r a Represents a match of multiple emoji in a string that were joined by zero-width-joiners (ZWJ/``\u200D``). This class is only used for emoji that are not "recommended for general interchange" (non-RGI) by Unicode.org. The data property of this class is always None. first_emoji_matchsecond_emoji_matchcCs||g|_ |dSr)r2_update)rrGrHrrrrs  zEmojiMatchZWJNonRGI.__init__cCs>tdd|jD|_|jdj|_|jdj|_d|_dS)Ncsr>rr?r@rrrrBrCz.EmojiMatchZWJNonRGI._update..r)r rDr2rrrrr!rrrrIs zEmojiMatchZWJNonRGI._updatenext_emoji_matchcCs|j||dSr)r2r9rI)rrKrrr_adds  zEmojiMatchZWJNonRGI._addN)r)r+r,r-r rrIrLrrrrr {s r c@s*eZdZUdZeed<eeefed<dS)r z A named tuple containing the matched string and its :class:`EmojiMatch` object if it is an emoji or a single character that is not a unicode emoji. charsvalueN)r)r+r,r-r/__annotations__rr rrrrr s r stringkeep_zwjrccst}tj}g}d}t|}g}||krd}||} ||vr3|d7}| tkr2|r2|t| | q| |vr|d} || } | |krb|| | vrb| |vrNn| || } | d7} | |krb|| | vsId| vr| d} ||| } t| || | }| d}d}|t| |ne| tkr|r|dj|vr|dkr||d|vr||||djdtj dkr|t d d |d d D}||tkr|d7}|d=n|d d =n |t|dj}|d=q|r|Ed Hg}|s| d kr| dkr|t| | |d7}||ks|Ed Hd S)a Finds unicode emoji in a string. Yields all normal characters as a named tuple :class:`Token` ``(char, char)`` and all emoji as :class:`Token` ``(chars, EmojiMatch)``. :param string: String contains unicode characters. MUST BE UNICODE. :param keep_zwj: Should ZWJ-characters (``\u200D``) that join non-RGI emoji be skipped or should be yielded as normal characters :return: An iterable of tuples :class:`Token` ``(char, char)`` or :class:`Token` ``(chars, EmojiMatch)`` rFr4rTrJstatus componentcss|]}t|jVqdSr)r6rM)rAtrrrrBsztokenize..Nu︎u️) get_search_treerr7r6r r9r r rMSTATUSsum)rPrQtreer7resultr:lengthignoreconsumedcharjsub_treer code_points match_objrrrr sv         Dr matches emoji_only join_emojic csJ|s |s |EdHdS|s|D] }|jtkr|VqdSd}d}d}g}|D]v}|}|r5|jtkr5d}q't|jtr|r|rt|djtrc|dj|jt|djt|j|dj|d<n%|}t|jtsoJ| t|jt|jt|j|jn| |d}d}q'd}d}|EdH|s|Vg}q'|EdHdS)ap Filters the output of `tokenize()` :param matches: An iterable of tuples of the form ``(match_str, result)`` where ``result`` is either an EmojiMatch or a string. :param emoji_only: If True, only EmojiMatch are returned in the output. If False all characters are returned :param join_emoji: If True, multiple EmojiMatch are merged into a single :class:`EmojiMatchZWJNonRGI` if they are separated only by a ZWJ. :return: An iterable of tuples :class:`Token` ``(char, char)``, :class:`Token` ``(chars, EmojiMatch)`` or :class:`Token` ``(chars, EmojiMatchZWJNonRGI)`` NFTrJ) rMr rN isinstancer r rLr popr9) rcrdretokenprevious_is_emojiprevious_is_zwjpre_previous_is_emoji accumulatorprevrrrrsX       rcCsdts0tjD]*}t}t|d}t|D]\}}||vri||<||}||kr.tj||d<qqtS)u Generate a search tree for demojize(). Example of a search tree:: EMOJI_DATA = {'a': {'en': ':Apple:'}, 'b': {'en': ':Bus:'}, 'ba': {'en': ':Bat:'}, 'band': {'en': ':Beatles:'}, 'bandit': {'en': ':Outlaw:'}, 'bank': {'en': ':BankOfEngland:'}, 'bb': {'en': ':BB-gun:'}, 'c': {'en': ':Car:'}} _SEARCH_TREE = {'a': {'data': {'en': ':Apple:'}}, 'b': {'a': {'data': {'en': ':Bat:'}, 'n': {'d': {'data': {'en': ':Beatles:'}, 'i': {'t': {'data': {'en': ':Outlaw:'}}}}, 'k': {'data': {'en': ':BankOfEngland:'}}}}, 'b': {'data': {'en': ':BB-gun:'}}, 'data': {'en': ':Bus:'}}, 'c': {'data': {'en': ':Car:'}}} _SEARCH_TREE / | ⧵ / | ⧵ a b c | / | ⧵ | | / | ⧵ | :Apple: ba :Bus: bb :Car: / ⧵ | / ⧵ | :Bat: ban :BB-gun: / ⧵ / ⧵ band bank / ⧵ | / ⧵ | bandi :Beatles: :BankOfEngland: | bandit | :Outlaw: r4r)rrr7r6 enumerate)emjr`lastidxr:r^rrrrV>s0  rVN)r-typingrrrrrrrr__all__r rr/rOr r r r r1r rrVrrrrs*   =$ [ E