ó Òbc@`sÅddlmZmZddlZddlZddlZddlZddlZddl Z ddl m Z ddl m Z yeWnek rŸeZnXddlZddlZddlmZmZmZmZgZeed„Zejdƒd„Zejd ƒejejZ e!e ƒZ"d „Z#ejd ƒd „Z$ejd ƒej%ddƒddd„Z&ejdƒej%ddƒddd„Z'ejdƒd„Z(ejdƒd„Z)ejdƒdded„Z+ejdƒddd„Z,ejd ƒdded!„Z-ejd"ƒddd#„Z.ejd$ƒdd%„Z/ejd&ƒded'„Z0ejd(ƒd)„Z1ejd*ƒd+„Z2e3d,„Z4ejd-ƒd.„Z5ejd/ƒd0„Z6ejd1ƒd2„Z7ejd3ƒd4„Z8ejd5ƒd6„Z9ejd7ƒyddl:Z:d8„Z;Wn&e<k rpddl=Z=d9„Z;nXejd:ƒyddl>Z>d;„Z?Wne<k r³d<„Z?nXejd=ƒdS(>i(tabsolute_importtdivisionN(tmd5(tdebug(tbase_urlencode_stringtbase_replace_nonprintablestbase_unicodisetbase_deunicodisecC`s˜|rt|ƒpt|ƒ}|rŠddddg}d}x)|dkrg|d}|jdƒ}q?W|rzt|ƒpƒt|ƒ|fS|dfSdS( NtKtMtGtTtiii(tfloattinttpop(tsizethuman_readabletfloating_pointtcoeffstcoeff((sH/oak/stanford/groups/akundaje/marinovg/programs/s3cmd-master/S3/Utils.pyt formatSize's "RcC`s3i}x&|D]}|d||djƒcs(R-(R&R'R(((sH/oak/stanford/groups/akundaje/marinovg/programs/s3cmd-master/S3/Utils.pyt mktmpfilebs R8cC`sdtƒ}tt|ƒdƒ:}x0trS|jdƒ}|sCPn|j|ƒq$WWdQX|jƒS(Ntrbi ii€(RR2R3tTruetreadtupdatet hexdigest(R6thtfptdata((sH/oak/stanford/groups/akundaje/marinovg/programs/s3cmd-master/S3/Utils.pyt hash_file_md5hs  RAcC`s|jtjƒ}g}xB|r\tjjttjj|ƒƒƒ r\|j|jƒƒqWx¯|r|j|jƒƒtjj|ƒ}y%t d|ƒtj t|ƒƒWq`t t fk rã}t d||j fƒtStk r }t d||fƒtSXq`WtS(s‘ mkdir_with_parents(dst_dir) Create directory 'dir_name' with all parent directories Returns True on success, False otherwise. s mkdir(%s)s(Can not make directory '%s' (Reason: %s)(tsplitR!tseptpathtisdirR3tjointappendRRR/R#tIOErrortstrerrortFalset ExceptionR:(tdir_namet pathmemberst tmp_stacktcur_dirR,((sH/oak/stanford/groups/akundaje/marinovg/programs/s3cmd-master/S3/Utils.pytmkdir_with_parentsus".  RPtreplacecC`s.|stjjƒj}nt||||ƒS(N(tS3tConfigtencodingR(tstringRTterrorstsilent((sH/oak/stanford/groups/akundaje/marinovg/programs/s3cmd-master/S3/Utils.pyt unicodisesRXcC`st|||tƒS(s. Alias to silent version of unicodise (RXR:(RURTRV((sH/oak/stanford/groups/akundaje/marinovg/programs/s3cmd-master/S3/Utils.pyt unicodise_s—sRYcC`s.|stjjƒj}nt||||ƒS(N(RRRSRTR(RURTRVRW((sH/oak/stanford/groups/akundaje/marinovg/programs/s3cmd-master/S3/Utils.pyR3ŸsR3cC`st|||tƒS(s0 Alias to silent version of deunicodise (R3R:(RURTRV((sH/oak/stanford/groups/akundaje/marinovg/programs/s3cmd-master/S3/Utils.pyt deunicodise_s¦sRZcC`s"tt||ƒ|ƒjddƒS(ss Convert 'string' to Unicode according to current encoding and replace all invalid characters with '?' u�t?(RXR3RQ(RURT((sH/oak/stanford/groups/akundaje/marinovg/programs/s3cmd-master/S3/Utils.pytunicodise_safe®sR\cC`s1|dkr!tjjƒj}nt|||ƒS(N(tNoneRRRSturlencoding_modeR(RUR^tunicode_output((sH/oak/stanford/groups/akundaje/marinovg/programs/s3cmd-master/S3/Utils.pyturlencode_string¹s R`cC`s%tjjƒjdk}t||ƒS(s replace_nonprintables(string) Replaces all non-printable characters 'ch' in 'string' where ord(ch) <= 26 with ^@, ^A, ... ^Z t fixbucket(RRRSR^R(RUtwarning_message((sH/oak/stanford/groups/akundaje/marinovg/programs/s3cmd-master/S3/Utils.pytreplace_nonprintablesÀsRccC`sXt|tƒr|St|tƒs4t|tjƒrGttj|ƒƒSt|dƒrottj|jƒƒƒSt|dƒr‘t|jdƒƒSt|t ƒs¾t|t ƒs¾t|t ƒr>y5|j dƒrètjƒt|dƒSt|ƒSWq>t k r:ytj|ƒSWq;t k r6}td|ƒq;Xq>Xntjjd|ƒ‚dS( s­Convert time specified in a variety of forms into UNIX epoch time. Accepts datetime.datetime, int, anything that has a strftime() method, and standard time 9-tuples t timetupletstrftimes%st+is&Failed to parse date with strptime: %ss}Unable to convert %r to an epoch time. Pass an epoch time. Try `date -d 'now + 1 year' +%%s` (shell) or time.mktime (Python).N(t isinstanceRRttimet struct_timetmktimethasattrRdRetstrtunicodetbytest startswitht ValueErrortstrptimeRRRt ExceptionstParameterError(tttex((sH/oak/stanford/groups/akundaje/marinovg/programs/s3cmd-master/S3/Utils.pyt time_to_epochÌs(!-   cC`sé|rPtjd|tjƒ}|r—tjjd||jƒdfƒ‚q—nGtjd|tjƒ}|r—tjjd||jƒdfƒ‚nt|ƒdkrÂtjjd|ƒ‚nt|ƒdkrítjjd |ƒ‚n|råt|ƒd krtjjd |ƒ‚ntjd |tjƒrOtjjd |ƒ‚ntjd|tjƒr€tjjd|ƒ‚ntjd|tjƒs±tjjd|ƒ‚ntjd|tjƒsåtjjd|ƒ‚qåntS(Ns([^a-z0-9\.-])s™Bucket name '%s' contains disallowed character '%s'. The only supported ones are: lowercase us-ascii letters (a-z), digits (0-9), dot (.) and hyphen (-).is([^A-Za-z0-9\._-])s¤Bucket name '%s' contains disallowed character '%s'. The only supported ones are: us-ascii letters (a-z, A-Z), digits (0-9), dot (.), hyphen (-) and underscore (_).is0Bucket name '%s' is too short (min 3 characters)iÿs1Bucket name '%s' is too long (max 255 characters)i?s0Bucket name '%s' is too long (max 63 characters)s-\.sEBucket name '%s' must not contain sequence '-.' for DNS compatibilitys\.\.sEBucket name '%s' must not contain sequence '..' for DNS compatibilitys ^[0-9a-z]s4Bucket name '%s' must start with a letter or a digits [0-9a-z]$s2Bucket name '%s' must end with a letter or a digit( tretsearchtUNICODERRRrRstgroupsRR:(tbuckett dns_stricttinvalid((sH/oak/stanford/groups/akundaje/marinovg/programs/s3cmd-master/S3/Utils.pytcheck_bucket_nameîs.,)R~cC`s3yt|dtƒSWntjjk r.tSXdS(NR|(R~R:RRRrRsRJ(R{((sH/oak/stanford/groups/akundaje/marinovg/programs/s3cmd-master/S3/Utils.pyt check_bucket_name_dns_conformity sRcC`sd|krtSt|ƒS(sk Check whether either the host_bucket support buckets and either bucket name is dns compatible s %(bucket)s(RJR(t bucket_hostt bucket_name((sH/oak/stanford/groups/akundaje/marinovg/programs/s3cmd-master/S3/Utils.pytcheck_bucket_name_dns_supports R‚cC`s€dtjjƒjkr"|tfStjjƒjjƒidd6}tj||tjƒ}|sm|tfS|jdƒt fS(sÎ bucket, success = getBucketFromHostname(hostname) Only works for hostnames derived from bucket names using Config.host_bucket pattern. Returns bucket name and a boolean success flag. s %(bucket)ss(?P.*)R{i( RRRSt host_bucketRJRRwtmatchRytgroupR:(thostnametpatterntm((sH/oak/stanford/groups/akundaje/marinovg/programs/s3cmd-master/S3/Utils.pytgetBucketFromHostnames  # R‰cC`s!tjjƒjjƒi|d6S(NR{(RRRSRƒR(R{((sH/oak/stanford/groups/akundaje/marinovg/programs/s3cmd-master/S3/Utils.pytgetHostnameFromBucket3sRŠcC`s”tƒ}|}|dkr}|j|ƒx_|dkry|jt||ƒƒ}|sYPn|j|ƒ|t|ƒ8}q+Wn |j|ƒ|jƒS(NR i(RtseekR;tminR<RR=(tbuffertmfiletoffsett chunk_sizet send_chunktmd5_hasht size_leftR@((sH/oak/stanford/groups/akundaje/marinovg/programs/s3cmd-master/S3/Utils.pytcalculateChecksum8s     R”cC`sttj|ƒjƒS(s>returns a username from the password databse for the given uid(RYtpwdtgetpwuidtpw_name(tuid((sH/oak/stanford/groups/akundaje/marinovg/programs/s3cmd-master/S3/Utils.pytgetpwuid_usernameMscC`sttjƒƒS(N(RYtgetpasstgetuser(R˜((sH/oak/stanford/groups/akundaje/marinovg/programs/s3cmd-master/S3/Utils.pyR™RsR™cC`sttj|ƒjƒS(s<returns a groupname from the group databse for the given gid(RYtgrptgetgrgidtgr_name(tgid((sH/oak/stanford/groups/akundaje/marinovg/programs/s3cmd-master/S3/Utils.pytgetgrgid_grpnameXscC`sdS(Nunobody((RŸ((sH/oak/stanford/groups/akundaje/marinovg/programs/s3cmd-master/S3/Utils.pyR \sR (@t __future__RRR!RhRwRUt string_modRR$thashlibRtloggingRRmt NameErrorRlt S3.ConfigRRt S3.Exceptionst S3.BaseUtilsRRRRt__all__RJRRGRt ascii_letterstdigitsRRRR R-tgetenvR0R8RARPR]RXRYR3RZR\R`RcRvR:R~RR‚R‰RŠR”R•R™t ImportErrorRšRœR (((sH/oak/stanford/groups/akundaje/marinovg/programs/s3cmd-master/S3/Utils.pyt s”          "                      "