tools/userland-fetch
author Norm Jacobs <Norm.Jacobs@Sun.COM>
Sun, 19 Sep 2010 15:50:53 -0500
changeset 34 d20b10eba317
parent 18 tools/fetch.py@6da6bd662127
child 42 566ce4d2ff99
permissions -rwxr-xr-x
prepare for zone based component builds
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
     1
#!/usr/bin/python2.6
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
     2
#
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
     3
# CDDL HEADER START
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
     4
#
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
     5
# The contents of this file are subject to the terms of the
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
     6
# Common Development and Distribution License (the "License").
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
     7
# You may not use this file except in compliance with the License.
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
     8
#
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
     9
# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    10
# or http://www.opensolaris.org/os/licensing.
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    11
# See the License for the specific language governing permissions
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    12
# and limitations under the License.
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    13
#
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    14
# When distributing Covered Code, include this CDDL HEADER in each
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    15
# file and include the License file at usr/src/OPENSOLARIS.LICENSE.
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    16
# If applicable, add the following below this CDDL HEADER, with the
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    17
# fields enclosed by brackets "[]" replaced with your own identifying
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    18
# information: Portions Copyright [yyyy] [name of copyright owner]
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    19
#
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    20
# CDDL HEADER END
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    21
#
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    22
# Copyright (c) 2010, Oracle and/or it's affiliates.  All rights reserved.
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    23
#
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    24
#
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    25
# fetch.py - a file download utility
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    26
#
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    27
#  A simple program similiar to wget(1), but handles local file copy, ignores
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    28
#  directories, and verifies file hashes.
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    29
#
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    30
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    31
import os
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    32
import sys
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    33
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    34
def validate(filename, hash):
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    35
	import hashlib
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    36
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    37
	try:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    38
		file = open(filename, 'r')
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    39
	except IOError:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    40
		return False
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    41
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    42
	if (hash == None):
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    43
		return True
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    44
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    45
	algorithm, value = hash.split(':')
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    46
	try:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    47
		m = hashlib.new(algorithm)
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    48
	except ValueError:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    49
		return False
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    50
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    51
	while True:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    52
		block = file.read()
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    53
		m.update(block)
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    54
		if block == '':
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    55
			break
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    56
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    57
	return m.hexdigest() == value
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    58
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    59
def download(url, filename = None, hash = None, verbose = False):
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    60
	from urllib import splittype, urlopen
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    61
	import hashlib
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    62
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    63
	src = None
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    64
	filename = None
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    65
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    66
	#print "Downloading %s from %s" % (filename, url)
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    67
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    68
	# open the download source
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    69
	if splittype(url)[0]:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    70
		try:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    71
			src = urlopen(url)
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    72
		except IOError:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    73
			return None
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    74
		remote_name = src.geturl().split('/')[-1]
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    75
	elif os.path.isfile(url):
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    76
		os.symlink(url, filename)
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    77
		return filename, filehash(filename)
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    78
		src = open(url, 'r')
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    79
		remote_name = url.split('/')[-1]
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    80
	else:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    81
		return None
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    82
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    83
	if filename == None:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    84
		filename = remote_name
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    85
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    86
	# if we have a source to download, open a destination and copy the data
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    87
	if src:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    88
		dst = open(filename, 'wb');
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    89
		if verbose:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    90
			print "Downloading %s from %s" % (filename, url)
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    91
		while True:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    92
			block = src.read()
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    93
			if block == '':
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    94
				break;
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    95
			dst.write(block)
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    96
		dst.close()
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    97
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    98
	# return the name of the file that we downloaded the data to.
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    99
	return filename
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   100
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   101
def download_paths(search, filename):
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   102
	tmp = os.getenv('DOWNLOAD_SEARCH_PATH')
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   103
	if tmp:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   104
		search += tmp.split(' ')
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   105
	base = os.path.basename(filename)
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   106
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   107
	urls = list()
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   108
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   109
	for path in search:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   110
		urls.append(path+'/'+base)
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   111
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   112
	return urls
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   113
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   114
def usage():
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   115
	print "Usage: %s [-v|--verbose] [-f|--file (file)] [-h|--hash (hash)] --url (url)" % (sys.argv[0].split('/')[-1])
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   116
	sys.exit(1)
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   117
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   118
def main():
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   119
	from urllib import splittype, urlopen
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   120
	import getopt
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   121
	import sys
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   122
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   123
	# FLUSH STDOUT 
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   124
	sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0)
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   125
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   126
	verbose = False
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   127
	filename = None
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   128
	url = None
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   129
	hash = None
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   130
	search = list()
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   131
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   132
	try:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   133
		opts, args = getopt.getopt(sys.argv[1:], "f:h:s:u:v",
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   134
			["file=", "hash=", "search=", "url=", "verbose"])
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   135
	except getopt.GetoptError, err:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   136
		print str(err)
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   137
		usage()
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   138
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   139
	for opt, arg in opts:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   140
		if opt in [ "-f", "--file" ]:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   141
			filename = arg
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   142
		elif opt in [ "-h", "--hash" ]:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   143
			hash = arg
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   144
		elif opt in [ "-s", "--search" ]:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   145
			search.append(arg)
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   146
		elif opt in [ "-u", "--url" ]:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   147
			url = arg
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   148
		elif opt in [ "-v", "--verbose" ]:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   149
			verbose = True
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   150
		else:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   151
			assert False, "unknown option"
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   152
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   153
	if url == None:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   154
		usage()
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   155
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   156
	print "Fetching %s" % filename
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   157
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   158
	if validate(filename, hash):
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   159
		print "\tcurrent copy is ok, nothing to do"
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   160
		sys.exit(0)
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   161
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   162
	# generate a list of URLS to search for a suitable download
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   163
	urls = download_paths(search, filename)
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   164
	urls.append(url)
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   165
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   166
	for url in urls:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   167
		print "\tTrying %s..." % url,
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   168
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   169
		if os.path.isfile(url):
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   170
			os.symlink(url, filename)
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   171
		elif splittype(url)[0]:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   172
			if download(url, filename) == None:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   173
				print "failed"
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   174
				continue
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   175
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   176
		print "retrieved...",
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   177
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   178
		if validate(filename, hash):
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   179
			print "validated"
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   180
			sys.exit(0)
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   181
		else:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   182
			print "corrupt"
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   183
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   184
		try:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   185
			os.remove(filename)
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   186
		except OSError:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   187
			pass
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   188
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   189
	sys.exit(1)
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   190
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   191
if __name__ == "__main__":
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   192
	main()