ó Òbc@@sŽddlmZddlmZddlmZddlmZddlmZddlmZmZddl m Z m Z m Z m Z dd lmZdd lmZdd lmZmZmZdd lZdd lZdd lZdd lZdd lZdd lZejdkZdddgZd„Zd„Zd„Z d„Z!d„Z"d„Z#e$d d„Z&e$d id„Z'd„Z(d S(i(tabsolute_importi(tS3(tConfig(tS3Uri(tFileDict(t dateS3toUnixtdateRFC822toUnix(t unicodiset deunicodiset deunicodise_streplace_nonprintables(tParameterError(t HashCache(tdebugtinfotwarningNitfetch_local_listtfetch_remote_listtcompare_filelistscc@sytjt|ƒƒ}Wntk r-dSXgg}}xp|D]h}t|ƒ}tjjttjj||ƒƒƒrt||ƒsª|j |ƒqªqB|j |ƒqBW|||fVxZ|D]R}tjj||ƒ}tjj t|ƒƒsÃxt |ƒD] }|VqWqÃqÃWdS(s_ Reimplementation of python's os.walk to nicely support unicode in input as in output. N( tostlistdirRt ExceptionRtpathtisdirtjointhandle_exclude_include_walk_dirtappendtislinkt_os_walk_unicode(ttoptnamestdirstnondirstnametnew_pathtx((sL/oak/stanford/groups/akundaje/marinovg/programs/s3cmd-master/S3/FileLists.pyR!s"    ' c@sLtƒ}d}tjj||dƒ}td|ƒt}xa|jD]V‰t‡fd†|DƒƒsmqEnˆj|ƒrEt }td|j ˆƒPqEqEW|r#x{|j D]m‰t‡fd †|Dƒƒs×q¯ntd |ˆj fƒˆj|ƒr¯t}td |j ˆƒPq¯q¯Wn|r:td |ƒntd |ƒ|S(s  Should this root/dirname directory be excluded? (otherwise included by default) Exclude dir matches in the current directory This prevents us from recursing down trees we know we want to ignore return True for excluding, and False for including u/)$u/)\Zu\/$u \/\Z(?ms)tu CHECK: %rc3@s!|]}ˆjj|ƒVqdS(N(tpatterntendswith(t.0tdp(tr(sL/oak/stanford/groups/akundaje/marinovg/programs/s3cmd-master/S3/FileLists.pys IsuEXCL-MATCH: '%s'c3@s!|]}ˆjj|ƒVqdS(N(R%R&(R'R((R)(sL/oak/stanford/groups/akundaje/marinovg/programs/s3cmd-master/S3/FileLists.pys SsuINCL-TEST: %s ~ %suINCL-MATCH: '%s'u EXCLUDE: %ruPASS: %r(u/)$u/)\Zu\/$u \/\Z(?ms)(RRRRR tFalsetexcludetanytsearchtTruet debug_excludetincludeR%t debug_include(troottdirnametcfgtdirectory_patternstdtexcluded((R)sL/oak/stanford/groups/akundaje/marinovg/programs/s3cmd-master/S3/FileLists.pyR:s2  c c@sPtjjt|ƒƒst‚t|gƒ}xÝt|ƒD]Ï\}}}ttjjt|ƒƒƒ}xŸ|D]—}tjj ||ƒ}ttjjt|ƒƒƒ}tjj t|ƒƒrn||ksä|j |tjj ƒrõt d|ƒq|j|ƒqnqnWq:Wx<|D]4} x+t| ƒD]\}}}|||fVq'WqWdS(sÑ Walk filesystem, following symbolic links (but without recursion), on python2.4 and later If a symlink directory loop is detected, emit a warning and skip. E.g.: dir1/dir2/sym-dir -> ../dir2 s+Skipping recursively symlinked directory %sN(RRRRtAssertionErrortsetRRtrealpathRRt startswithtsepRtadd( Rtwalkdirstdirpathtdirnamest filenamest real_dirpathR3tcurrentt real_currenttwalkdir((sL/oak/stanford/groups/akundaje/marinovg/programs/s3cmd-master/S3/FileLists.pyt_fswalk_follow_symlinksbs   cc@s2x+t|ƒD]\}}}|||fVq WdS(s\ Directory tree generator path (str) is the root of the directory tree to walk N(R(RR?R@RA((sL/oak/stanford/groups/akundaje/marinovg/programs/s3cmd-master/S3/FileLists.pyt_fswalk_no_symlinkszscC@s!tdƒtƒ}tdtƒ}xò|jƒD]ä}td|ƒt}x?|jD]4}|j|ƒrSt}td|j|ƒPqSqSW|rÖxB|j D]4}|j|ƒr›t}td|j |ƒPq›q›Wn|rtd|ƒ||||<||=q/q/td|ƒq/W||fS(NuApplying --exclude/--includet ignore_caseu CHECK: %suEXCL-MATCH: '%s'uINCL-MATCH: '%s'u EXCLUDE: %suPASS: %r( R RRR*tkeysR+R-R.R/R0R1(tsrc_listR4t exclude_listtfileR7R)((sL/oak/stanford/groups/akundaje/marinovg/programs/s3cmd-master/S3/FileLists.pytfilter_exclude_includeƒs0   c C@s¡d„}i}xD|jD]9}zd}|dkr[tjtjjƒdddtƒ}nOytjt|ƒddƒ}Wn-t k r©}t d||j fƒwnXx|D]…}t |ƒj ƒ}tjjtjj||ƒƒ}t tjjt|ƒƒƒ}t tjjt|ƒƒƒ} |||| ƒq±WWd|rQ|jƒnXqWg} x>t|ƒD]0} || } | jƒ| j| g| fƒqiW| S(NcS@s1||kr|g|| %s)u CHECK: %suIGNR: %s (used --skip-existing)uIGNR: %s (disappeared)u%s: file disappeared, ignoring.uIGNR: %s (transfer not needed)uDST COPY src: %s -> %suDST COPY dst: %s -> %s(R*R8RRRR RIt skip_existingRZRxRRƒRTtby_md5t find_md5_oneRRÍ( RJRïRéRêRçRðt update_listt copy_pairsR‰t same_fileRttdst1Re((R4RïsL/oak/stanford/groups/akundaje/marinovg/programs/s3cmd-master/S3/FileLists.pyRsr '               (ii()t __future__RRRRRt BaseUtilsRRtUtilsRRR R t ExceptionsR R tloggingR RRRRWRÙRÆRyRUt version_infoRŸt__all__RRRFRGRMRjR*RTRRR(((sL/oak/stanford/groups/akundaje/marinovg/programs/s3cmd-master/S3/FileLists.pyt s4"        (   &¬•