tools/userland-fetch
author Srinivasa Sarva <srinivasa.sarva@oracle.com>
Tue, 15 Feb 2011 16:12:40 -0800
changeset 90 5483a3df388e
parent 58 d8024c042a00
child 135 5c6feb0253fa
permissions -rwxr-xr-x
7019432 Move tcpdump to userland
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
     1
#!/usr/bin/python2.6
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
     2
#
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
     3
# CDDL HEADER START
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
     4
#
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
     5
# The contents of this file are subject to the terms of the
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
     6
# Common Development and Distribution License (the "License").
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
     7
# You may not use this file except in compliance with the License.
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
     8
#
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
     9
# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    10
# or http://www.opensolaris.org/os/licensing.
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    11
# See the License for the specific language governing permissions
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    12
# and limitations under the License.
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    13
#
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    14
# When distributing Covered Code, include this CDDL HEADER in each
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    15
# file and include the License file at usr/src/OPENSOLARIS.LICENSE.
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    16
# If applicable, add the following below this CDDL HEADER, with the
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    17
# fields enclosed by brackets "[]" replaced with your own identifying
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    18
# information: Portions Copyright [yyyy] [name of copyright owner]
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    19
#
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    20
# CDDL HEADER END
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    21
#
58
d8024c042a00 7011369 gmake complains about fresh userland bits
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 42
diff changeset
    22
# Copyright (c) 2010, 2011, Oracle and/or its affiliates. All rights reserved.
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    23
#
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    24
#
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    25
# fetch.py - a file download utility
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    26
#
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    27
#  A simple program similiar to wget(1), but handles local file copy, ignores
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    28
#  directories, and verifies file hashes.
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    29
#
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    30
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    31
import os
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    32
import sys
42
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
    33
import shutil
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
    34
from urllib import splittype, urlopen
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    35
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    36
def validate(filename, hash):
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    37
	import hashlib
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    38
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    39
	try:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    40
		file = open(filename, 'r')
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    41
	except IOError:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    42
		return False
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    43
58
d8024c042a00 7011369 gmake complains about fresh userland bits
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 42
diff changeset
    44
	algorithm, hashvalue = hash.split(':')
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    45
	try:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    46
		m = hashlib.new(algorithm)
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    47
	except ValueError:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    48
		return False
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    49
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    50
	while True:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    51
		block = file.read()
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    52
		m.update(block)
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    53
		if block == '':
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    54
			break
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    55
58
d8024c042a00 7011369 gmake complains about fresh userland bits
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 42
diff changeset
    56
	return "%s:%s" % (algorithm, m.hexdigest())
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    57
42
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
    58
def download(url, filename = None):
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    59
	src = None
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    60
42
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
    61
	try:
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
    62
		src = urlopen(url)
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
    63
	except IOError:
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    64
		return None
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    65
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    66
	if filename == None:
42
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
    67
		filename = src.geturl().split('/')[-1]
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    68
42
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
    69
	try:
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    70
		dst = open(filename, 'wb');
42
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
    71
	except IOError:
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
    72
		src.close()
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
    73
		return None
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
    74
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
    75
	while True:
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
    76
		block = src.read()
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
    77
		if block == '':
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
    78
			break;
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
    79
		dst.write(block)
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
    80
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
    81
	src.close()
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
    82
	dst.close()
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    83
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    84
	# return the name of the file that we downloaded the data to.
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    85
	return filename
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    86
42
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
    87
def download_paths(search, filename, url):
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    88
	urls = list()
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    89
42
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
    90
	if filename != None:
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
    91
		tmp = os.getenv('DOWNLOAD_SEARCH_PATH')
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
    92
		if tmp:
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
    93
			search += tmp.split(' ')
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
    94
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
    95
		file = os.path.basename(filename)
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
    96
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
    97
		urls = [ base + '/' + file for base in search ]
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
    98
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
    99
		# filename should always be first
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   100
		if filename in urls:
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   101
			urls.remove(filename)
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   102
		urls.insert(0, filename)
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   103
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   104
	# command line url is a fallback, so it's last
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   105
	if url != None and url not in urls:
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   106
		urls.append(url)
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   107
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   108
	return urls
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   109
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   110
def usage():
42
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   111
	print "Usage: %s [-f|--file (file)] [-l|--link] [-h|--hash (hash)] [-s|--search (search-dir)] --url (url)" % (sys.argv[0].split('/')[-1])
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   112
	sys.exit(1)
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   113
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   114
def main():
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   115
	import getopt
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   116
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   117
	# FLUSH STDOUT 
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   118
	sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0)
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   119
42
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   120
	file_arg = None
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   121
	link_arg = False
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   122
	hash_arg = None
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   123
	url_arg = None
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   124
	search_list = list()
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   125
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   126
	try:
42
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   127
		opts, args = getopt.getopt(sys.argv[1:], "f:h:ls:u:",
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   128
			["file=", "link", "hash=", "search=", "url="])
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   129
	except getopt.GetoptError, err:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   130
		print str(err)
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   131
		usage()
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   132
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   133
	for opt, arg in opts:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   134
		if opt in [ "-f", "--file" ]:
42
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   135
			file_arg = arg
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   136
		elif opt in [ "-l", "--link" ]:
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   137
			link_arg = True
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   138
		elif opt in [ "-h", "--hash" ]:
42
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   139
			hash_arg = arg
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   140
		elif opt in [ "-s", "--search" ]:
42
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   141
			search_list.append(arg)
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   142
		elif opt in [ "-u", "--url" ]:
42
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   143
			url_arg = arg
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   144
		else:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   145
			assert False, "unknown option"
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   146
42
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   147
	if url_arg == None:
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   148
		usage()
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   149
42
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   150
	for url in download_paths(search_list, file_arg, url_arg):
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   151
		print "Source %s..." % url,
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   152
42
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   153
		scheme, path = splittype(url)
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   154
		name = file_arg
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   155
42
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   156
		if scheme in [ None, 'file' ]:
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   157
			if os.path.exists(path) == False:
58
d8024c042a00 7011369 gmake complains about fresh userland bits
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 42
diff changeset
   158
				print "not found, skipping file copy"
42
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   159
				continue
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   160
			elif name != path:
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   161
				if link_arg == False:
58
d8024c042a00 7011369 gmake complains about fresh userland bits
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 42
diff changeset
   162
					print "\n    copying..."
42
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   163
					shutil.copy2(path, name)
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   164
				else:
58
d8024c042a00 7011369 gmake complains about fresh userland bits
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 42
diff changeset
   165
					print "\n    linking..."
42
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   166
					os.symlink(path, name)
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   167
			else:
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   168
				pass
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   169
		elif scheme in [ 'http', 'https', 'ftp' ]:
58
d8024c042a00 7011369 gmake complains about fresh userland bits
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 42
diff changeset
   170
			print "\n    downloading...",
42
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   171
			name = download(url, file_arg)
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   172
			if name == None:
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   173
				print "failed"
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   174
				continue
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   175
58
d8024c042a00 7011369 gmake complains about fresh userland bits
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 42
diff changeset
   176
		print "\n    validating...",
d8024c042a00 7011369 gmake complains about fresh userland bits
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 42
diff changeset
   177
		if hash_arg == None:
d8024c042a00 7011369 gmake complains about fresh userland bits
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 42
diff changeset
   178
			print "skipping (no hash)"
d8024c042a00 7011369 gmake complains about fresh userland bits
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 42
diff changeset
   179
			sys.exit(0)
d8024c042a00 7011369 gmake complains about fresh userland bits
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 42
diff changeset
   180
			
d8024c042a00 7011369 gmake complains about fresh userland bits
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 42
diff changeset
   181
		realhash = validate(name, hash_arg)
d8024c042a00 7011369 gmake complains about fresh userland bits
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 42
diff changeset
   182
		if realhash == hash_arg:
42
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   183
			print "ok"
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   184
			sys.exit(0)
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   185
		else:
58
d8024c042a00 7011369 gmake complains about fresh userland bits
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 42
diff changeset
   186
			print "corruption detected"
d8024c042a00 7011369 gmake complains about fresh userland bits
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 42
diff changeset
   187
			print "    expected: %s" % hash_arg
d8024c042a00 7011369 gmake complains about fresh userland bits
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 42
diff changeset
   188
			print "    actual:   %s" % realhash
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   189
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   190
		try:
42
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   191
			os.remove(name)
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   192
		except OSError:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   193
			pass
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   194
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   195
	sys.exit(1)
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   196
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   197
if __name__ == "__main__":
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   198
	main()