tools/userland-fetch
author Mike Sullivan <Mike.Sullivan@Oracle.COM>
Thu, 17 Dec 2015 17:50:05 -0800
changeset 5213 68d34c2581d6
parent 4429 a6c5fd1cbbc9
child 5244 c63fce595432
permissions -rwxr-xr-x
Close of build 90.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
3770
ca450a806cc1 20440888 Userland python tools should migrate to 2.7
John Beck <John.Beck@Oracle.COM>
parents: 3533
diff changeset
     1
#!/usr/bin/python2.7
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
     2
#
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
     3
# CDDL HEADER START
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
     4
#
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
     5
# The contents of this file are subject to the terms of the
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
     6
# Common Development and Distribution License (the "License").
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
     7
# You may not use this file except in compliance with the License.
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
     8
#
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
     9
# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    10
# or http://www.opensolaris.org/os/licensing.
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    11
# See the License for the specific language governing permissions
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    12
# and limitations under the License.
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    13
#
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    14
# When distributing Covered Code, include this CDDL HEADER in each
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    15
# file and include the License file at usr/src/OPENSOLARIS.LICENSE.
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    16
# If applicable, add the following below this CDDL HEADER, with the
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    17
# fields enclosed by brackets "[]" replaced with your own identifying
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    18
# information: Portions Copyright [yyyy] [name of copyright owner]
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    19
#
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    20
# CDDL HEADER END
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    21
#
3770
ca450a806cc1 20440888 Userland python tools should migrate to 2.7
John Beck <John.Beck@Oracle.COM>
parents: 3533
diff changeset
    22
# Copyright (c) 2010, 2015, Oracle and/or its affiliates. All rights reserved.
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    23
#
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    24
#
3533
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    25
# userland-fetch - a file download utility
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    26
#
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    27
#  A simple program similiar to wget(1), but handles local file copy, ignores
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    28
#  directories, and verifies file hashes.
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    29
#
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    30
3533
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    31
import errno
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    32
import os
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    33
import sys
42
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
    34
import shutil
3533
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    35
import subprocess
832
d0946a4ddb78 PSARC/2012/198 python-jsonrpclib
Edward Pilatowicz <edward.pilatowicz@oracle.com>
parents: 800
diff changeset
    36
from urllib import splittype
d0946a4ddb78 PSARC/2012/198 python-jsonrpclib
Edward Pilatowicz <edward.pilatowicz@oracle.com>
parents: 800
diff changeset
    37
from urllib2 import urlopen
135
5c6feb0253fa 7028115 userland-fetch fails to validate gd2 archive
Norm Jacobs <Norm.Jacobs@Oracle.COM>
parents: 58
diff changeset
    38
import hashlib
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    39
136
6a7dd3a5aaf9 7027131 userland-fetch should be more verbose when printing IOError exception
Vladimir Marek <Vladimir.Marek@oracle.com>
parents: 135
diff changeset
    40
def printIOError(e, txt):
6a7dd3a5aaf9 7027131 userland-fetch should be more verbose when printing IOError exception
Vladimir Marek <Vladimir.Marek@oracle.com>
parents: 135
diff changeset
    41
	""" Function to decode and print IOError type exception """
6a7dd3a5aaf9 7027131 userland-fetch should be more verbose when printing IOError exception
Vladimir Marek <Vladimir.Marek@oracle.com>
parents: 135
diff changeset
    42
	print "I/O Error: " + txt + ": "
6a7dd3a5aaf9 7027131 userland-fetch should be more verbose when printing IOError exception
Vladimir Marek <Vladimir.Marek@oracle.com>
parents: 135
diff changeset
    43
	try:
6a7dd3a5aaf9 7027131 userland-fetch should be more verbose when printing IOError exception
Vladimir Marek <Vladimir.Marek@oracle.com>
parents: 135
diff changeset
    44
		(code, message) = e
6a7dd3a5aaf9 7027131 userland-fetch should be more verbose when printing IOError exception
Vladimir Marek <Vladimir.Marek@oracle.com>
parents: 135
diff changeset
    45
		print str(message) + " (" + str(code) + ")"
6a7dd3a5aaf9 7027131 userland-fetch should be more verbose when printing IOError exception
Vladimir Marek <Vladimir.Marek@oracle.com>
parents: 135
diff changeset
    46
	except:
6a7dd3a5aaf9 7027131 userland-fetch should be more verbose when printing IOError exception
Vladimir Marek <Vladimir.Marek@oracle.com>
parents: 135
diff changeset
    47
		print str(e)
3533
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    48
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    49
def validate_signature(path, signature):
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    50
	"""Given paths to a file and a detached PGP signature, verify that
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    51
	the signature is valid for the file.  Current configuration allows for
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    52
	unrecognized keys to be downloaded as necessary."""
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    53
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    54
	# Find the root of the repo so that we can point GnuPG at the right
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    55
	# configuration and keyring.
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    56
	proc = subprocess.Popen(["hg", "root"], stdout=subprocess.PIPE)
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    57
	proc.wait()
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    58
	if proc.returncode != 0:
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    59
		return False
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    60
	out, err = proc.communicate()
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    61
	gpgdir = os.path.join(out.strip(), "tools", ".gnupg")
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    62
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    63
        # Skip the permissions warning: none of the information here is private,
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    64
        # so not having to worry about getting mercurial keeping the directory
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    65
        # unreadable is just simplest.
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    66
	try:
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    67
		proc = subprocess.Popen(["gpg2", "--verify",
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    68
		    "--no-permission-warning", "--homedir", gpgdir, signature,
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    69
		    path], stdin=open("/dev/null"),
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    70
		    stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    71
	except OSError as e:
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    72
		# If the executable simply couldn't be found, just skip the
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    73
		# validation.
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    74
		if e.errno == errno.ENOENT:
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    75
			return False
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    76
		raise
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    77
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    78
        proc.wait()
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    79
        if proc.returncode != 0:
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    80
		# Only print GnuPG's output when there was a problem.
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    81
                print proc.stdout.read()
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    82
                return False
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    83
        return True
136
6a7dd3a5aaf9 7027131 userland-fetch should be more verbose when printing IOError exception
Vladimir Marek <Vladimir.Marek@oracle.com>
parents: 135
diff changeset
    84
	
135
5c6feb0253fa 7028115 userland-fetch fails to validate gd2 archive
Norm Jacobs <Norm.Jacobs@Oracle.COM>
parents: 58
diff changeset
    85
def validate(file, hash):
3533
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    86
	"""Given a file-like object and a hash string, verify that the hash
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    87
	matches the file contents."""
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    88
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    89
	try:
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    90
		algorithm, hashvalue = hash.split(':')
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    91
	except:
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
    92
		algorithm = "sha256"
800
2ad056ed89ec 7163771 sha1 should not be used in userland-fetch and userland component/*/Makefile
Mike Sullivan <Mike.Sullivan@Oracle.COM>
parents: 212
diff changeset
    93
2ad056ed89ec 7163771 sha1 should not be used in userland-fetch and userland component/*/Makefile
Mike Sullivan <Mike.Sullivan@Oracle.COM>
parents: 212
diff changeset
    94
	# force migration away from sha1
2ad056ed89ec 7163771 sha1 should not be used in userland-fetch and userland component/*/Makefile
Mike Sullivan <Mike.Sullivan@Oracle.COM>
parents: 212
diff changeset
    95
	if algorithm == "sha1":
2ad056ed89ec 7163771 sha1 should not be used in userland-fetch and userland component/*/Makefile
Mike Sullivan <Mike.Sullivan@Oracle.COM>
parents: 212
diff changeset
    96
		algorithm = "sha256"
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    97
	try:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    98
		m = hashlib.new(algorithm)
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
    99
	except ValueError:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   100
		return False
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   101
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   102
	while True:
137
6fb20db3eee4 7028115 userland-fetch fails to validate gd2 archive (fix bzip)
Norm Jacobs <Norm.Jacobs@Oracle.COM>
parents: 136
diff changeset
   103
		try:
6fb20db3eee4 7028115 userland-fetch fails to validate gd2 archive (fix bzip)
Norm Jacobs <Norm.Jacobs@Oracle.COM>
parents: 136
diff changeset
   104
			block = file.read()
6fb20db3eee4 7028115 userland-fetch fails to validate gd2 archive (fix bzip)
Norm Jacobs <Norm.Jacobs@Oracle.COM>
parents: 136
diff changeset
   105
		except IOError, err:
6fb20db3eee4 7028115 userland-fetch fails to validate gd2 archive (fix bzip)
Norm Jacobs <Norm.Jacobs@Oracle.COM>
parents: 136
diff changeset
   106
			print str(err),
6fb20db3eee4 7028115 userland-fetch fails to validate gd2 archive (fix bzip)
Norm Jacobs <Norm.Jacobs@Oracle.COM>
parents: 136
diff changeset
   107
			break
6fb20db3eee4 7028115 userland-fetch fails to validate gd2 archive (fix bzip)
Norm Jacobs <Norm.Jacobs@Oracle.COM>
parents: 136
diff changeset
   108
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   109
		m.update(block)
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   110
		if block == '':
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   111
			break
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   112
58
d8024c042a00 7011369 gmake complains about fresh userland bits
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 42
diff changeset
   113
	return "%s:%s" % (algorithm, m.hexdigest())
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   114
135
5c6feb0253fa 7028115 userland-fetch fails to validate gd2 archive
Norm Jacobs <Norm.Jacobs@Oracle.COM>
parents: 58
diff changeset
   115
def validate_container(filename, hash):
3533
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   116
	"""Given a file path and a hash string, verify that the hash matches the
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   117
	file contents."""
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   118
135
5c6feb0253fa 7028115 userland-fetch fails to validate gd2 archive
Norm Jacobs <Norm.Jacobs@Oracle.COM>
parents: 58
diff changeset
   119
	try:
5c6feb0253fa 7028115 userland-fetch fails to validate gd2 archive
Norm Jacobs <Norm.Jacobs@Oracle.COM>
parents: 58
diff changeset
   120
		file = open(filename, 'r')
136
6a7dd3a5aaf9 7027131 userland-fetch should be more verbose when printing IOError exception
Vladimir Marek <Vladimir.Marek@oracle.com>
parents: 135
diff changeset
   121
	except IOError as e:
6a7dd3a5aaf9 7027131 userland-fetch should be more verbose when printing IOError exception
Vladimir Marek <Vladimir.Marek@oracle.com>
parents: 135
diff changeset
   122
		printIOError(e, "Can't open file " + filename)
135
5c6feb0253fa 7028115 userland-fetch fails to validate gd2 archive
Norm Jacobs <Norm.Jacobs@Oracle.COM>
parents: 58
diff changeset
   123
		return False
5c6feb0253fa 7028115 userland-fetch fails to validate gd2 archive
Norm Jacobs <Norm.Jacobs@Oracle.COM>
parents: 58
diff changeset
   124
	return validate(file, hash)
5c6feb0253fa 7028115 userland-fetch fails to validate gd2 archive
Norm Jacobs <Norm.Jacobs@Oracle.COM>
parents: 58
diff changeset
   125
5c6feb0253fa 7028115 userland-fetch fails to validate gd2 archive
Norm Jacobs <Norm.Jacobs@Oracle.COM>
parents: 58
diff changeset
   126
5c6feb0253fa 7028115 userland-fetch fails to validate gd2 archive
Norm Jacobs <Norm.Jacobs@Oracle.COM>
parents: 58
diff changeset
   127
def validate_payload(filename, hash):
3533
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   128
	"""Given a file path and a hash string, verify that the hash matches the
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   129
	payload (uncompressed content) of the file."""
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   130
135
5c6feb0253fa 7028115 userland-fetch fails to validate gd2 archive
Norm Jacobs <Norm.Jacobs@Oracle.COM>
parents: 58
diff changeset
   131
	import re
5c6feb0253fa 7028115 userland-fetch fails to validate gd2 archive
Norm Jacobs <Norm.Jacobs@Oracle.COM>
parents: 58
diff changeset
   132
	import gzip
5c6feb0253fa 7028115 userland-fetch fails to validate gd2 archive
Norm Jacobs <Norm.Jacobs@Oracle.COM>
parents: 58
diff changeset
   133
	import bz2
5c6feb0253fa 7028115 userland-fetch fails to validate gd2 archive
Norm Jacobs <Norm.Jacobs@Oracle.COM>
parents: 58
diff changeset
   134
5c6feb0253fa 7028115 userland-fetch fails to validate gd2 archive
Norm Jacobs <Norm.Jacobs@Oracle.COM>
parents: 58
diff changeset
   135
	expr_bz = re.compile('.+\.bz2$', re.IGNORECASE)
5c6feb0253fa 7028115 userland-fetch fails to validate gd2 archive
Norm Jacobs <Norm.Jacobs@Oracle.COM>
parents: 58
diff changeset
   136
	expr_gz = re.compile('.+\.gz$', re.IGNORECASE)
800
2ad056ed89ec 7163771 sha1 should not be used in userland-fetch and userland component/*/Makefile
Mike Sullivan <Mike.Sullivan@Oracle.COM>
parents: 212
diff changeset
   137
	expr_tgz = re.compile('.+\.tgz$', re.IGNORECASE)
135
5c6feb0253fa 7028115 userland-fetch fails to validate gd2 archive
Norm Jacobs <Norm.Jacobs@Oracle.COM>
parents: 58
diff changeset
   138
5c6feb0253fa 7028115 userland-fetch fails to validate gd2 archive
Norm Jacobs <Norm.Jacobs@Oracle.COM>
parents: 58
diff changeset
   139
	try:
5c6feb0253fa 7028115 userland-fetch fails to validate gd2 archive
Norm Jacobs <Norm.Jacobs@Oracle.COM>
parents: 58
diff changeset
   140
		if expr_bz.match(filename):
137
6fb20db3eee4 7028115 userland-fetch fails to validate gd2 archive (fix bzip)
Norm Jacobs <Norm.Jacobs@Oracle.COM>
parents: 136
diff changeset
   141
			file = bz2.BZ2File(filename, 'r')
135
5c6feb0253fa 7028115 userland-fetch fails to validate gd2 archive
Norm Jacobs <Norm.Jacobs@Oracle.COM>
parents: 58
diff changeset
   142
		elif expr_gz.match(filename):
5c6feb0253fa 7028115 userland-fetch fails to validate gd2 archive
Norm Jacobs <Norm.Jacobs@Oracle.COM>
parents: 58
diff changeset
   143
			file = gzip.GzipFile(filename, 'r')
800
2ad056ed89ec 7163771 sha1 should not be used in userland-fetch and userland component/*/Makefile
Mike Sullivan <Mike.Sullivan@Oracle.COM>
parents: 212
diff changeset
   144
		elif expr_tgz.match(filename):
2ad056ed89ec 7163771 sha1 should not be used in userland-fetch and userland component/*/Makefile
Mike Sullivan <Mike.Sullivan@Oracle.COM>
parents: 212
diff changeset
   145
			file = gzip.GzipFile(filename, 'r')
135
5c6feb0253fa 7028115 userland-fetch fails to validate gd2 archive
Norm Jacobs <Norm.Jacobs@Oracle.COM>
parents: 58
diff changeset
   146
		else:
5c6feb0253fa 7028115 userland-fetch fails to validate gd2 archive
Norm Jacobs <Norm.Jacobs@Oracle.COM>
parents: 58
diff changeset
   147
			return False
136
6a7dd3a5aaf9 7027131 userland-fetch should be more verbose when printing IOError exception
Vladimir Marek <Vladimir.Marek@oracle.com>
parents: 135
diff changeset
   148
	except IOError as e:
6a7dd3a5aaf9 7027131 userland-fetch should be more verbose when printing IOError exception
Vladimir Marek <Vladimir.Marek@oracle.com>
parents: 135
diff changeset
   149
		printIOError(e, "Can't open archive " + filename)
135
5c6feb0253fa 7028115 userland-fetch fails to validate gd2 archive
Norm Jacobs <Norm.Jacobs@Oracle.COM>
parents: 58
diff changeset
   150
		return False
5c6feb0253fa 7028115 userland-fetch fails to validate gd2 archive
Norm Jacobs <Norm.Jacobs@Oracle.COM>
parents: 58
diff changeset
   151
	return validate(file, hash)
5c6feb0253fa 7028115 userland-fetch fails to validate gd2 archive
Norm Jacobs <Norm.Jacobs@Oracle.COM>
parents: 58
diff changeset
   152
5c6feb0253fa 7028115 userland-fetch fails to validate gd2 archive
Norm Jacobs <Norm.Jacobs@Oracle.COM>
parents: 58
diff changeset
   153
3533
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   154
def download(url, filename=None, quiet=False):
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   155
	"""Download the content at the given URL to the given filename
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   156
	(defaulting to the basename of the URL if not given.  If 'quiet' is
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   157
	True, throw away any error messages.  Returns the name of the file to
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   158
	which the content was donloaded."""
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   159
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   160
	src = None
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   161
42
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   162
	try:
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   163
		src = urlopen(url)
136
6a7dd3a5aaf9 7027131 userland-fetch should be more verbose when printing IOError exception
Vladimir Marek <Vladimir.Marek@oracle.com>
parents: 135
diff changeset
   164
	except IOError as e:
3533
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   165
		if not quiet:
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   166
			printIOError(e, "Can't open url " + url)
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   167
		return None
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   168
212
8a66c0c0e8dc 7041047 userland-fetch should handle http errors more gracefully
Vladimir Marek <Vladimir.Marek@oracle.com>
parents: 137
diff changeset
   169
	# 3xx, 4xx and 5xx (f|ht)tp codes designate unsuccessfull action
8a66c0c0e8dc 7041047 userland-fetch should handle http errors more gracefully
Vladimir Marek <Vladimir.Marek@oracle.com>
parents: 137
diff changeset
   170
	if 3 <= int(src.getcode()/100) <= 5:
3533
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   171
		if not quiet:
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   172
			print "Error code: " + str(src.getcode())
212
8a66c0c0e8dc 7041047 userland-fetch should handle http errors more gracefully
Vladimir Marek <Vladimir.Marek@oracle.com>
parents: 137
diff changeset
   173
		return None
8a66c0c0e8dc 7041047 userland-fetch should handle http errors more gracefully
Vladimir Marek <Vladimir.Marek@oracle.com>
parents: 137
diff changeset
   174
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   175
	if filename == None:
42
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   176
		filename = src.geturl().split('/')[-1]
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   177
42
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   178
	try:
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   179
		dst = open(filename, 'wb');
136
6a7dd3a5aaf9 7027131 userland-fetch should be more verbose when printing IOError exception
Vladimir Marek <Vladimir.Marek@oracle.com>
parents: 135
diff changeset
   180
	except IOError as e:
3533
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   181
		if not quiet:
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   182
			printIOError(e, "Can't open file " + filename + " for writing")
42
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   183
		src.close()
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   184
		return None
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   185
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   186
	while True:
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   187
		block = src.read()
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   188
		if block == '':
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   189
			break;
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   190
		dst.write(block)
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   191
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   192
	src.close()
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   193
	dst.close()
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   194
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   195
	# return the name of the file that we downloaded the data to.
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   196
	return filename
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   197
42
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   198
def download_paths(search, filename, url):
3533
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   199
	"""Returns a list of URLs where the file 'filename' might be found,
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   200
	using 'url', 'search', and $DOWNLOAD_SEARCH_PATH as places to look.
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   201
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   202
	If 'filename' is None, then the list will simply contain 'url'.
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   203
	"""
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   204
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   205
	urls = list()
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   206
42
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   207
	if filename != None:
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   208
		tmp = os.getenv('DOWNLOAD_SEARCH_PATH')
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   209
		if tmp:
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   210
			search += tmp.split(' ')
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   211
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   212
		file = os.path.basename(filename)
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   213
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   214
		urls = [ base + '/' + file for base in search ]
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   215
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   216
		# filename should always be first
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   217
		if filename in urls:
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   218
			urls.remove(filename)
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   219
		urls.insert(0, filename)
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   220
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   221
	# command line url is a fallback, so it's last
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   222
	if url != None and url not in urls:
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   223
		urls.append(url)
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   224
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   225
	return urls
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   226
3533
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   227
def download_from_paths(search_list, file_arg, url, link_arg, quiet=False):
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   228
	"""Attempts to download a file from a number of possible locations.
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   229
	Generates a list of paths where the file ends up on the local
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   230
	filesystem.  This is a generator because while a download might be
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   231
	successful, the signature or hash may not validate, and the caller may
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   232
	want to try again from the next location.  The 'link_arg' argument is a
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   233
	boolean which, when True, specifies that if the source is not a remote
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   234
	URL and not already found where it should be, to make a symlink to the
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   235
	source rather than copying it.
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   236
	"""
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   237
	for url in download_paths(search_list, file_arg, url):
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   238
		if not quiet:
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   239
			print "Source %s..." % url,
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   240
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   241
		scheme, path = splittype(url)
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   242
		name = file_arg
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   243
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   244
		if scheme in [ None, 'file' ]:
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   245
			if os.path.exists(path) == False:
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   246
				if not quiet:
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   247
					print "not found, skipping file copy"
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   248
				continue
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   249
			elif name and name != path:
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   250
				if link_arg == False:
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   251
					if not quiet:
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   252
						print "\n    copying..."
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   253
					shutil.copy2(path, name)
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   254
				else:
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   255
					if not quiet:
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   256
						print "\n    linking..."
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   257
					os.symlink(path, name)
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   258
		elif scheme in [ 'http', 'https', 'ftp' ]:
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   259
			if not quiet:
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   260
				print "\n    downloading...",
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   261
			name = download(url, file_arg, quiet)
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   262
			if name == None:
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   263
				if not quiet:
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   264
					print "failed"
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   265
				continue
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   266
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   267
		yield name
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   268
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   269
def usage():
3533
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   270
	print "Usage: %s [-f|--file (file)] [-l|--link] [-h|--hash (hash)] " \
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   271
          "[-s|--search (search-dir)] [-S|--sigurl (signature-url)] --url (url)" % \
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   272
          (sys.argv[0].split('/')[-1])
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   273
	sys.exit(1)
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   274
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   275
def main():
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   276
	import getopt
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   277
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   278
	# FLUSH STDOUT 
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   279
	sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0)
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   280
42
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   281
	file_arg = None
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   282
	link_arg = False
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   283
	hash_arg = None
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   284
	url_arg = None
3533
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   285
	sig_arg = None
42
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   286
	search_list = list()
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   287
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   288
	try:
3533
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   289
                opts, args = getopt.getopt(sys.argv[1:], "f:h:ls:S:u:",
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   290
			["file=", "link", "hash=", "search=", "sigurl=", "url="])
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   291
	except getopt.GetoptError, err:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   292
		print str(err)
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   293
		usage()
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   294
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   295
	for opt, arg in opts:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   296
		if opt in [ "-f", "--file" ]:
42
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   297
			file_arg = arg
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   298
		elif opt in [ "-l", "--link" ]:
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   299
			link_arg = True
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   300
		elif opt in [ "-h", "--hash" ]:
42
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   301
			hash_arg = arg
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   302
		elif opt in [ "-s", "--search" ]:
42
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   303
			search_list.append(arg)
3533
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   304
		elif opt in [ "-S", "--sigurl" ]:
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   305
			sig_arg = arg
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   306
		elif opt in [ "-u", "--url" ]:
42
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   307
			url_arg = arg
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   308
		else:
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   309
			assert False, "unknown option"
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   310
3533
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   311
	for name in download_from_paths(search_list, file_arg, url_arg, link_arg):
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   312
		print "\n    validating signature...",
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   313
3533
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   314
		sig_valid = False
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   315
		if not sig_arg:
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   316
			print "skipping (no signature URL)"
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   317
		else:
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   318
			# Put the signature file in the same directory as the
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   319
			# file we're downloading.
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   320
			sig_file = os.path.join(
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   321
			    os.path.dirname(file_arg),
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   322
			    os.path.basename(sig_arg))
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   323
			# Validate with the first signature we find.
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   324
			for sig_file in download_from_paths(search_list, sig_file,
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   325
			    sig_arg, link_arg, True):
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   326
				if sig_file:
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   327
					if validate_signature(name, sig_file):
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   328
						print "ok"
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   329
						sig_valid = True
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   330
					else:
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   331
						print "failed"
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   332
					break
42
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   333
				else:
3533
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   334
					continue
42
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   335
			else:
3533
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   336
				print "failed (couldn't fetch signature)"
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   337
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   338
		print "    validating hash...",
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   339
		realhash = validate_container(name, hash_arg)
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   340
3533
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   341
		if not hash_arg:
58
d8024c042a00 7011369 gmake complains about fresh userland bits
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 42
diff changeset
   342
			print "skipping (no hash)"
3533
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   343
			print "hash is: %s" % realhash
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   344
		elif realhash == hash_arg:
42
566ce4d2ff99 fix using/downloading shared archives
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents: 34
diff changeset
   345
			print "ok"
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   346
		else:
135
5c6feb0253fa 7028115 userland-fetch fails to validate gd2 archive
Norm Jacobs <Norm.Jacobs@Oracle.COM>
parents: 58
diff changeset
   347
			payloadhash = validate_payload(name, hash_arg)
5c6feb0253fa 7028115 userland-fetch fails to validate gd2 archive
Norm Jacobs <Norm.Jacobs@Oracle.COM>
parents: 58
diff changeset
   348
			if payloadhash == hash_arg:
5c6feb0253fa 7028115 userland-fetch fails to validate gd2 archive
Norm Jacobs <Norm.Jacobs@Oracle.COM>
parents: 58
diff changeset
   349
				print "ok"
3533
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   350
			else:
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   351
				# If the signature validated, then we assume
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   352
				# that the expected hash is just a typo, but we
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   353
				# warn just in case.
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   354
				if sig_valid:
4429
a6c5fd1cbbc9 21132783 userland-fetch should fail if the hash fails, even if the signature doesn't
Danek Duvall <danek.duvall@oracle.com>
parents: 4196
diff changeset
   355
					print "invalid hash! Did you forget " \
a6c5fd1cbbc9 21132783 userland-fetch should fail if the hash fails, even if the signature doesn't
Danek Duvall <danek.duvall@oracle.com>
parents: 4196
diff changeset
   356
					    "to update it?"
3533
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   357
				else:
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   358
					print "corruption detected"
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   359
3533
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   360
				print "    expected: %s" % hash_arg
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   361
				print "    actual:   %s" % realhash
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   362
				print "    payload:  %s" % payloadhash
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   363
4429
a6c5fd1cbbc9 21132783 userland-fetch should fail if the hash fails, even if the signature doesn't
Danek Duvall <danek.duvall@oracle.com>
parents: 4196
diff changeset
   364
				# If the hash is invalid, but the signature
a6c5fd1cbbc9 21132783 userland-fetch should fail if the hash fails, even if the signature doesn't
Danek Duvall <danek.duvall@oracle.com>
parents: 4196
diff changeset
   365
				# validation succeeded, rename the archive (so
a6c5fd1cbbc9 21132783 userland-fetch should fail if the hash fails, even if the signature doesn't
Danek Duvall <danek.duvall@oracle.com>
parents: 4196
diff changeset
   366
				# the user doesn't have to re-download it) and
a6c5fd1cbbc9 21132783 userland-fetch should fail if the hash fails, even if the signature doesn't
Danek Duvall <danek.duvall@oracle.com>
parents: 4196
diff changeset
   367
				# fail.  Otherwise, try to remove the file and
a6c5fd1cbbc9 21132783 userland-fetch should fail if the hash fails, even if the signature doesn't
Danek Duvall <danek.duvall@oracle.com>
parents: 4196
diff changeset
   368
				# try again.
a6c5fd1cbbc9 21132783 userland-fetch should fail if the hash fails, even if the signature doesn't
Danek Duvall <danek.duvall@oracle.com>
parents: 4196
diff changeset
   369
				if sig_valid:
a6c5fd1cbbc9 21132783 userland-fetch should fail if the hash fails, even if the signature doesn't
Danek Duvall <danek.duvall@oracle.com>
parents: 4196
diff changeset
   370
					newname = name + ".invalid-hash"
a6c5fd1cbbc9 21132783 userland-fetch should fail if the hash fails, even if the signature doesn't
Danek Duvall <danek.duvall@oracle.com>
parents: 4196
diff changeset
   371
					try:
a6c5fd1cbbc9 21132783 userland-fetch should fail if the hash fails, even if the signature doesn't
Danek Duvall <danek.duvall@oracle.com>
parents: 4196
diff changeset
   372
						os.rename(name, newname)
a6c5fd1cbbc9 21132783 userland-fetch should fail if the hash fails, even if the signature doesn't
Danek Duvall <danek.duvall@oracle.com>
parents: 4196
diff changeset
   373
					except OSError:
a6c5fd1cbbc9 21132783 userland-fetch should fail if the hash fails, even if the signature doesn't
Danek Duvall <danek.duvall@oracle.com>
parents: 4196
diff changeset
   374
						pass
a6c5fd1cbbc9 21132783 userland-fetch should fail if the hash fails, even if the signature doesn't
Danek Duvall <danek.duvall@oracle.com>
parents: 4196
diff changeset
   375
					else:
a6c5fd1cbbc9 21132783 userland-fetch should fail if the hash fails, even if the signature doesn't
Danek Duvall <danek.duvall@oracle.com>
parents: 4196
diff changeset
   376
						print "archive saved as %s; " \
a6c5fd1cbbc9 21132783 userland-fetch should fail if the hash fails, even if the signature doesn't
Danek Duvall <danek.duvall@oracle.com>
parents: 4196
diff changeset
   377
						    "if it isn't corrupt, " \
a6c5fd1cbbc9 21132783 userland-fetch should fail if the hash fails, even if the signature doesn't
Danek Duvall <danek.duvall@oracle.com>
parents: 4196
diff changeset
   378
						    "rename to %s" % (newname,
a6c5fd1cbbc9 21132783 userland-fetch should fail if the hash fails, even if the signature doesn't
Danek Duvall <danek.duvall@oracle.com>
parents: 4196
diff changeset
   379
						    name)
a6c5fd1cbbc9 21132783 userland-fetch should fail if the hash fails, even if the signature doesn't
Danek Duvall <danek.duvall@oracle.com>
parents: 4196
diff changeset
   380
					sys.exit(1)
a6c5fd1cbbc9 21132783 userland-fetch should fail if the hash fails, even if the signature doesn't
Danek Duvall <danek.duvall@oracle.com>
parents: 4196
diff changeset
   381
				else:
3533
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   382
					try:
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   383
						os.remove(name)
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   384
					except OSError:
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   385
						pass
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   386
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   387
					continue
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   388
0b8107a40da7 20183619 userland should support PGP signatures
Danek Duvall <danek.duvall@oracle.com>
parents: 832
diff changeset
   389
		sys.exit(0)
18
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   390
	sys.exit(1)
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   391
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   392
if __name__ == "__main__":
6da6bd662127 Add a fetch utility to do more than just download like wget does
Norm Jacobs <Norm.Jacobs@Sun.COM>
parents:
diff changeset
   393
	main()