author | Rich Burridge <rich.burridge@oracle.com> |
Mon, 02 Feb 2015 11:55:58 -0800 | |
changeset 3716 | a2629a2cf270 |
parent 3640 | ebe894a8833e |
child 3744 | a74b6fa1af7a |
permissions | -rw-r--r-- |
3640
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
1 |
Summary. |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
2 |
-------- |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
3 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
4 |
Testing of a userland component that provides tests is performed by hooking |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
5 |
those tests up to the 'test' target and running 'gmake test'. This should |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
6 |
generally pass, as failing tests may indicate things you have to fix, or |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
7 |
upstream tests that aren't applicable or need modifications on Solaris. |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
8 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
9 |
'gmake test' is often run when a component is upgraded or otherwise |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
10 |
intentionally changed, but sometimes it would be useful to rerun the tests |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
11 |
after something else has changed (such as the system being upgraded, or a |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
12 |
change in compilers) and see if that has affected the tests. |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
13 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
14 |
We do this by having a 'master test file' that contains the expected results, |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
15 |
and having a compare target that runs the tests and compares them with the |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
16 |
results. |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
17 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
18 |
Note: because the initial test run and the current test run may have a |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
19 |
different environment (different users, different locales, different machines, |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
20 |
different compilers...) the results need to have all such output dependencies |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
21 |
removed or abstracted. |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
22 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
23 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
24 |
Setting up a master test file for a Userland component. |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
25 |
------------------------------------------------------- |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
26 |
|
3716
a2629a2cf270
20451511 Yet more Userland components should have master test results to compare against
Rich Burridge <rich.burridge@oracle.com>
parents:
3640
diff
changeset
|
27 |
When setting up a test-and-compare run for a new component, you will need to |
a2629a2cf270
20451511 Yet more Userland components should have master test results to compare against
Rich Burridge <rich.burridge@oracle.com>
parents:
3640
diff
changeset
|
28 |
have master test file(s) that are identical for both the x86 and SPARC |
a2629a2cf270
20451511 Yet more Userland components should have master test results to compare against
Rich Burridge <rich.burridge@oracle.com>
parents:
3640
diff
changeset
|
29 |
platforms. It is suggested that they should initially be created by doing |
a2629a2cf270
20451511 Yet more Userland components should have master test results to compare against
Rich Burridge <rich.burridge@oracle.com>
parents:
3640
diff
changeset
|
30 |
something like the following in your x86 Userland workspace, then copied to |
a2629a2cf270
20451511 Yet more Userland components should have master test results to compare against
Rich Burridge <rich.burridge@oracle.com>
parents:
3640
diff
changeset
|
31 |
the same locations in your SPARC workspace and retested there. |
3640
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
32 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
33 |
When you run "gmake test", a check is made to see if there is a master |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
34 |
file of test results. If there is, then a test-then-compare run is performed. |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
35 |
If there isn't, then just a "normal" run of the test suite is performed. |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
36 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
37 |
The name of the master test file (or files to be exact), will depend upon |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
38 |
whether you have 32-bit, 64-bit, 32-and-64-bit and whether this is for a |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
39 |
Python or Perl component. |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
40 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
41 |
The default master file name is defined in |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
42 |
make-rules/shared-macros.mk and is: |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
43 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
44 |
COMPONENT_TEST_MASTER = $(COMPONENT_TEST_RESULTS_DIR)/results-$(BITS).master |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
45 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
46 |
where COMPONENT_TEST_RESULTS_DIR is: |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
47 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
48 |
COMPONENT_TEST_RESULTS_DIR = $(COMPONENT_DIR)/test |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
49 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
50 |
so that means it will default to looking for the following test file |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
51 |
master names: |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
52 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
53 |
32-bit: components/<component-name>/test/results-32.master |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
54 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
55 |
64-bit: components/<component-name>/test/results-64.master |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
56 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
57 |
both: components/<component-name>/test/results-32.master |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
58 |
components/<component-name>/test/results-64.master |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
59 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
60 |
For Python, COMPONENT_TEST_MASTER is overridden in |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
61 |
make-rules/setup.py.mk to be: |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
62 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
63 |
COMPONENT_TEST_MASTER = $(COMPONENT_TEST_RESULTS_DIR)/results-$(PYTHON_VERSION)-$(BITS).master |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
64 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
65 |
so that means it's looking for one or more of: |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
66 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
67 |
2.6: components/python/<component-name>/test/results-2.6-32.master |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
68 |
2.6: components/python/<component-name>/test/results-2.6-64.master |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
69 |
2.7: components/python/<component-name>/test/results-2.7-32.master |
3716
a2629a2cf270
20451511 Yet more Userland components should have master test results to compare against
Rich Burridge <rich.burridge@oracle.com>
parents:
3640
diff
changeset
|
70 |
2.7: components/python/<component-name>/test/results-2.7-64.master |
3640
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
71 |
3.4: components/python/<component-name>/test/results-3.4-64.master |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
72 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
73 |
depending upon which versions of Python this component supports. |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
74 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
75 |
Perl is similar, with COMPONENT_TEST_MASTER being overridden in: |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
76 |
make-rules/makemaker.mk to be: |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
77 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
78 |
COMPONENT_TEST_MASTER = $(COMPONENT_TEST_RESULTS_DIR)/results-$(PERL_VERSION)-$(BITS).master |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
79 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
80 |
so that means it's looking for one or more of: |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
81 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
82 |
5.12: components/perl_modules/<component>/test/results-5.12-32.master |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
83 |
5.12-mt: components/perl_modules/<component>/test/results-5.12-mt-32.master |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
84 |
5.16: components/perl_modules/<component>/test/results-5.16-64.master |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
85 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
86 |
depending upon which versions of Perl this component supports. |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
87 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
88 |
Note that if the test results are the same for both 32-bit and 64-bit or |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
89 |
for all versions of Python or Perl, you can override the |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
90 |
COMPONENT_TEST_MASTER definition in your component Makefile and just supply |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
91 |
a single master files file. For example, in components/python/pep8 Makefile |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
92 |
we have: |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
93 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
94 |
COMPONENT_TEST_MASTER = $(COMPONENT_TEST_RESULTS_DIR)/results-all.master |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
95 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
96 |
In order to do a test-then-compare run rather than just run the component |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
97 |
test suite, initially just create an empty master test file (or files). |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
98 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
99 |
For example, for elinks, which just has 64-bit tests, do: |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
100 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
101 |
$ cd components/elinks |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
102 |
$ mkdir test |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
103 |
$ touch test/results-64.master |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
104 |
$ gmake test |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
105 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
106 |
At this point, you have a set of test results in |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
107 |
components/elinks/test/test-64-results. |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
108 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
109 |
Even better, there are a set of "global" regular expressions that are |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
110 |
run on those test results to try to normalize them. The output from that |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
111 |
is placed in components/elinks/test/results-64.snapshot |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
112 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
113 |
You can now use the contents of that file as a first cut at the master results. |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
114 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
115 |
$ cp test/results-64.snapshot test/results-64.master |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
116 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
117 |
Now run the tests again. Note that you have to get back to a clean start |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
118 |
just in case the test process tries to compile any code. |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
119 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
120 |
$ gmake clean |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
121 |
$ gmake test |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
122 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
123 |
At this point, it will again compare the test results against the master(s), |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
124 |
and if there are still differences, they will be placed in |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
125 |
components/elinks/test/test-64-diffs |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
126 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
127 |
Typically these differences will be for things like usernames, temporary |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
128 |
filenames, timings etc. If you have some differences, then you are going |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
129 |
to have to write special regexp expressions in your component Makefile to |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
130 |
"normalize" them, and adjust the master test results file so that it |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
131 |
matches what the normalized version looks like. |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
132 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
133 |
For example, see the transform in the asciidoc Makefile: |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
134 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
135 |
COMPONENT_TEST_TRANSFORMS += \ |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
136 |
'-e "s|/tmp......|/tmpxxxxxx|g" ' |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
137 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
138 |
to "normalize" any temporary filenames that appear in the test results. |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
139 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
140 |
There will be other examples as more components are adjusted to test |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
141 |
against master results files. |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
142 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
143 |
If a lot of people start generating the very same ones, then we can |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
144 |
consider adding them to the "global" set of regexps in |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
145 |
make-rules/shared-macros.mk which currently looks like: |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
146 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
147 |
COMPONENT_TEST_TRANSFORMS = \ |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
148 |
'-e "s|$(@D)|\\$$(@D)|g" ' \ |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
149 |
'-e "s|$(PERL)|\\$$(PERL)|g" ' \ |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
150 |
'-e "s|$(SOURCE_DIR)|\\$$(SOURCE_DIR)|g" ' |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
151 |
|
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
152 |
When your master test file(s) are in good shape, then you should "hg add" |
ebe894a8833e
20359322 Need documentation for adding master test results to Userland components
Rich Burridge <rich.burridge@oracle.com>
parents:
diff
changeset
|
153 |
them to your workspace. |