AHjdZddlZddlmZddlmZej eZdede de dzd e dzd e e f d Z dd e e de de dzd e dzd e de d e e fdZy)z% Utilities dedicated to URL sampling N)sample)UrlStoreurlstore samplesize exclude_min exclude_maxreturnc :g}|jD]}|j|Dcgc] }|jdvr|j"}}|r |t ||ks|/t ||kDr!t j d|t |t ||kDrtt||}n|}|j|Dcgc]}||z c}t jd|t |t |t |z |Scc}wcc}w)z4Iterate through the hosts in store and draw samples.)/Nzdiscarded (size): %s urls: %s)kz%s urls: %s prop.: %s) urldict _load_urlsurlpathpathlenLOGGERwarningsortedrextenddebug) rrrr output_urlsdomainpurlpathsmysamples H/root/tools/cai/cai_env/lib/python3.12/site-packages/courlan/sampling.py _make_samplersK"" ((0 yy , FFH  &H +&H + NN=vs8} U  x=: %fX<=HH91FQJ9: '  M MCM )  - 8 7 (:s %D D input_urlsstrictverbosec|r$tjtjn#tjtjt dd||}|j |t||||S)zRSample a list of URLs by domain name, optionally using constraints on their numberTN) compressedlanguager r!)rsetLevelloggingDEBUGERRORradd_urlsr)rrrr r r!rs r sample_urlsr*6sV & &4$vwWH j! *k; GG)NNFF)__doc__r&randomrrr getLogger__name__rintliststrrboolr*r+rr5s   8 $$$$t$t $  #Y $T#" HS HHtHt H  H  H #YHr+