1.1 D-file

The first step is to understand what D-file is. It is simple text file with some rules to accommodate records and fields. Input next data using your favorite editor, and store it in a file named "supplier.d". (You may name it without ".d" or with any other extension, but it is recommendable to use consistent extension).

(1-1)

s#:1
sname:Smith
loc:London

s#:2
sname:Jones
loc:Paris

s#:3
sname:Blake
loc:Paris

s#:4
sname:Clark
loc:London

s#:5
sname:Adams
loc:Athens

This is a D-file. There are five D-records separated by a null line (line without any character). Each D-record has three fields named "s#", "sname" and "loc".

Here, one thing should be noted. Do not put SPACE after COLON (:). Spaces after COLON are regarded as a part of the value. If you write loc: London, the value is not "London" but " London". This D-file convention may cause some uneasiness for people who are used to modern writing styles. But, it makes the system much simpler. For example, we don't need special way to write values led by a SPACE character(s).

As the first step, try

(1-2)

Dpr supplier.d

Dpr is one of the most frequently used D-commands. It prints the contents of the input D-file int a table style to your terminal.

(1-2 output)

                                  supplier.d                                  1

rec# s# sname loc
   1  1 Smith London
   2  2 Jones Paris
   3  3 Blake Paris
   4  4 Clark London
   5  5 Adams Athens

D-file has no "schema definition". You can put any field in any format as long as it is represented by a character string. Validatation can be done with many ways using various D-commands. Dfd is simple and most frequently used tool. It reports the field attributes in D-files. Try next command.

(1-3)

Dfd supplier.d | Dpr -f exists:r -g filename

It will produce the following output. (In Linux, you can use just Dfdp supplier.d).

(1-3 output)

                  filename:supplier.d

rec# fieldname min max exists minlen maxlen avglen attribute
   1 s#          1   1    5/5      1      1    1   Int
   2 sname       1   1    5/5      5      5    5   Asc
   3 loc         1   1    5/5      5      6    5.6 Asc

The field min and max reports minimum and maximum occurence of the field, respectively. Both values should be "1" for a "normalized" file. The field exists value 5/5" reports that the field apears in five records out of total five records. (If min is not zero, it is always 100%). The field minlen, maxlen and avglen reports minimum, maximum and average length (number of characters) of the values, respectively. The last field attribute value Int means all the values are unsigned decimal integer, and the value Asc means ASCII character string. For other attributes, see the manual of Dfd.

Usually, you will detect your input error easily with Dfd (except for simple typos).

1.2 Reading lines

Next, we are going to make new D-file which has four fields named p#, pname, color, weight and qoh. We can input it using editor as the first file. But, we introduce a new way -- DfromLine. Create next file (with an editor or any other way), and name it "part.txt".

(1-4)

1→Nut→Red→12→30
2→Bolt→Green→17→60
3→Screw→Blue→17→30
4→Screw→Red→14→10
5→Cam→Blue→12→20
6→Cog→Red→19→40

The RIGHTWARDS ARROW (→) character here denotes a TAB character. Then, a command

(1-5)

DfromLine p#,pname,color,weight,qoh part.txt > part.d

reads lines form "part.txt" and produce a D-file part.d. The parameter p#,pname,color,weight,qoh is called field-list which gives the field names. Field names in a field-list are separated either by COMMA (,) or by SPACE (U+0020). Then, Dpr part.d produces next output.

(1-6)

              part.d

rec# p# pname color weight qoh
   1  1 Nut   Red       12  30
   2  2 Bolt  Green     17  60
   3  3 Screw Blue      17  30
   4  4 Screw Red       14  10
   5  5 Cam   Blue      12  20
   6  6 Cog   Red       19  40

To make the third file "project.d", we use slightly different way. Try next command.

(1-7)

DfromLine -t ", *" j#,jname,mgr_e# > project.d

Your terminal goes into input mode. Type in the following lines.

(1-7 input)

1, Sorter, 640028
2, Punch, 700128
3, Reader, 680096
4, Console, 660118
5, Collator, 690002
6, Terminal, 640014
7, tape, 710020

After the last line (after ENTER), type in CONTROL-D (in UNIX), or CONTROL-Z ENTER (in Windows). (These characters are called EOF character. They may be altered with some means. But, default setting is usually these.)

The option -t ", *" gives the delimiter pattern ( regular expression). This pattern means "a COMMA followed by zero or more SPACE characters". Command

(1-8)

Dpr project.d

produces next output.

(1-8 output)

        project.d

rec# j# jname    mgr_e#
   1  1 Sorter   640028
   2  2 Punch    700128
   3  3 Reader   680096
   4  4 Console  660118
   5  5 Collator 690002
   6  6 Terminal 640014
   7  7 tape     710020

1.3 Csv file and D-file

For the fourth D-file, we take another approach. Use your favorite spreadsheet (e.g., EXCEL) to input the next table, save the sheet as CSV file named "supply.csv".
(Of course, you may use any other tool, including editor, to make the csv file).

(1-9)

supply.csv

s#	p#	j#	shipdate	quantity
1	1	1	70,1,12	200
1	1	4	70,1,12	700
2	3	2	70,1,17	200
2	3	3	70,1,17	200
2	3	4	70,1,17	500
2	3	5	70,1,17	600
2	3	6	70,1,17	400
2	3	7	70,1,17	800
2	5	2	69,12,19	100
3	3	1	70,2,2	200
3	4	2	70,2,10	500
4	6	3	70,1,31	300
4	6	7	70,1,31	300
5	2	1	70,1,23	100
5	2	2	70,1,23	200
5	2	3	70,1,23	100
5	2	4	70,1,23	100
5	2	5	70,1,23	100
5	2	6	70,1,23	100
5	2	7	70,1,23	100
5	3	7	70,1,31	100
5	5	5	70,1,12	500
5	5	7	70,1,12	100
5	6	2	70,1,16	200
5	1	4	70,2,9	100
5	3	4	70,1,14	200
5	4	4	70,1,24	800
5	5	4	70,1,12	400
5	6	4	70,2,5	500

Make sure the delimiter is COMMA (,) and texts are quoted with QUOTATION MARK ("). Then, command:

(1-10)

DfromCsv supply.csv > supply.d

produces the D-file "supply.d". With DfromCsv, field names are always taken from the first row. If you use other character as the delimiter (e.g., TAB), you can use -t option to give the delimiter character. Note that you may need some special character to input TAB depending on your tty settings.

There is another D-command DtoCsv, which, naturally, converts D-file to CSV file. With these commands you can import and export your D-files to spreadsheet software.

1.4 D-file operations

You have created, in the previous section, "supplier.d", "part.d", "project.d" and "supply.d". These files are classic example files use by E. F. Codd when he introduced a database sublanguage "alpha". He showed some relational operations on these "relations". We are going to follow his examples to show D-file operations.

2.1 Handmade D-file and Dfdp

Handmade files inevitably involve errors. At least, those who prepare data should work this in mind. The file supply-handmade.d is based on supply.d in the 1.4.1 Spj world, and some artificial errors are added. You know the file. It should have three id-number fieds ( s#, p#, j#) and two fields describing supply transaction ( quantity and shipdate).

Try Dfdp (or Dfd and Dpr as in (1-3)).

(2-1)

Dfdp supply-handmade.d

It will produce the following output.

(2-1 output)

                           filename:supply-handmade.d

rec# fieldname min max exists minlen maxlen avglen  attribute
   1 s#          1   1  25/25      1      1 1       Int
   2 p#          1   1  25/25      1      1 1       Int
   3 j#          1   1  25/25      1      1 1       Int
   4 shipdata    0   1   1/25      7      7 7       Asc
   5 shipdate    0   1  24/25      6      8 6.95833 Asc
   6 quantity    1   1  25/25      3      3 3       Num

You will be easily aware that the fieldname shipdata is an error. The fact that the field shipdata exists only one in twenty records (1/25) strongly suggests that is an error.

You should also be aware that the attribute value Num in the field quantity also suggests an error. Because, it should be Int if the quantity is recorded correctly. You can use

(2-2)

Dselect quantity UNLIKE "^[0-9]+$" supply-handmade.d

to find the error record(s).

Actually, supply-handmade.d has another error in the field shipdate. But, it is not found with Dfdp. If minlen were smaller than 6 or maxlen were greater than 8, you could find the error. These numbers in the output from (2-1) is normal.

The field shipdate has a date as its value. Format of the date is rather unusual YY,M,D. As D has no special validation tool, you have to write validation program for Dselect. A makeshift version will be:

(2-3)

Dselect shipdate UNLIKE "^[0-9][0-9],1?[0-9],[123]?[0-9]$" supply-handmade.d

This program checks only character class and number of characters. With this one, no error is found. It is possible to make a Dl program for full check of date. It is bit longer to be coded in the command arguments. We make Dl file and use it with -f option. The Dl file will be:

(2-4)

COUNT shipdate EQ 1 AND (
    FIELD shipdate UNLIKE ^([0-9][0-9]),(1?[0-9]),([1-3]?[0-9])$ OR (
        VAR y = MATCH1 ;
        VAR m = MATCH2 ;
        VAR d = MATCH3 ;
        ( NUM VAR m LT 1 OR NUM VAR m GT 12 ) OR
        ( NUM VAR d LT 1 OR NUM VAR d GT 31 ) OR
        (
            ( NUM { 4 6 9 11 } INCL NUM VAR m ) AND
            VAR d GT 30
        ) OR
        ( NUM VAR m EQ 2 AND NUM VAR d GT 29 ) OR
        ( VAR y EQ 00 AND NUM VAR m EQ 2 AND NUM VAR d GT 28 ) OR
        ( INT VAR y % 4 NE 0 AND NUM VAR m EQ 2 AND NUM VAR d GT 28 )
    )
)

Note that, in a Dl file, you don't need quoatations unless you use REVERSE SOLIDUS (\), QUOTATION MARK (") or APOSTROPHE (') in a Dl word. This program uses AND and OR as the meaning of if-then statements. condition AND ( foo ) means IF condition THEN foo, and condition OR ( foo ) means IF NOT condition THEN foo. This type of usage of logical operators is common in shell or perl scripts. Then you can use this program with Dselect.

(2-5)

Dselect -f check-shipdate.dl supply-handmade.d

It will produce next output.

(2-5 output)

s#:5
p#:1
j#:4
shipdate:70,2,29
quantity:100

2.2 Reading a log file

Recently, you need not read log files by yourself, as major applications come with proprietary log analyzer. But, occasionally, you may have to read a log file. D will be a handy tool for reading log files.

The file websmpl.log is a portion of a log file from a web server. At a look, you will find that you cannot read this file with simple DfromLine shown in 1.2 Reading Lines, because it is not tab separated. In websmpl.log, fields are basically separated by a SPACE, but the time stamp field is enclosed with SQUARE BRACKETs embracing SPACEs in it. The HTTP-request field is enclosed with QUOTATION MARKs also empracing SPACEs.

Actually, DfromLine accepts field-format-list, which is an extension of a simple field-list and enables to read these fields. Format specification is placed after field-name separated by COLON.

(2-6)

DfromLine -t " " "host,logname,user,time:+1/] /,request:q,status,bytes,etime" websmpl.log

The option -t " " gives the default delimiter which is applied to the fields without delimter specification. In this case it is applied to the fields except for request.

Format specification +1/] / in the field time means "Skip one character (+1), then read the field delimited by RIGHT SQUARE BRACKET + SPACE (/] /).". The reason we need to skip the first character is just to exclude the first LEFT SQUARE BRACKET ([). Without +1 you will get the value with '[' and without ']'.

The delimiter for the particular field is given within //.

Special characters in the D-format is cumbersome subject. The detailed specification is described in a section of the manual D_fmt.

The format of the field request is q, which means "Read a string quoted by QUOTATION MARK (").".

It is good idea to check newly created D-files with Dfdp.

(2-7)

DfromLine -t " " "host,logname,user,time:+1/] /,request:q,status,bytes,etime" websmpl.log | \
Dfdp

It will produce the following output.

(2-7 output)

rec# fieldname min max exists minlen maxlen avglen attribute
   1 host        1   1    8/8     12     15 13.5   Asc
   2 logname     1   1    8/8      1      1  1     Asc
   3 user        1   1    8/8      1      1  1     Asc
   4 time        1   1    8/8     26     26 26     Asc
   5 request     1   1    8/8     14     37 30.375 Asc
   6 status      1   1    8/8      3      9  3.75  Asc
   7 bytes       1   1    8/8      1      4  3.375 Asc
   8 etime       1   1    8/8      2      4  2.25  Int

You may think nothing wrong with this output, but, if you know the field status should be three digit by HTTP specification, you will find the field has some problem. To check the field you will try

(2-8)

DfromLine -t " " "host,logname,user,time:+1/] /,request:q,status,bytes,etime" websmpl.log | \
Dselect f UNLIKE "^[0-9][0-9][0-9]$"

The output is

(2-8 output)

host:247.25.64.162
logname:-
user:-
time:17/Oct/2002:15:19:12 +0900
request:GET /cgi-bin/p03?id=B4903411\
status:HTTP/1.1"
bytes:200
etime:7217

From this and the original file websmpl.log you will find the problem is \" within the field e. This is UNIX type escape method to use QUOTATION MARK within quoted string. As D-format default is Microsoft CSV like escape method (using ""), you have to use format qx to read this log.

(2-9)

DfromLine -t " " "host,logname,user,time:+1/] /,request:qx,status,bytes,etime" websmpl.log

Before closing this section, I would like to introduce a good practice for checking this type of DfromLine usage. Use the following procedure instead of (2-9).

(2-10)

DfromLine -t " " \
"host,logname,user,time:+1/] /,request:qx,status,bytes,etime,XXX" \
websmpl.log | \
Dfdp

The point is adding excess field XXX. This field should not appear in the Dfdp output if all goes well. If the field XXX appears, or the last field etime is missing in some record (you can know this by exists field of the Dfdp output), you can know that something is wrong with the reading process.

2.3 Some more about input format

In the previous subsection, we used some features of D-format, such as positioning (+1), field individual delimiter (/] /) and quoting option (qx). In this subsection, we learn some other features.

First, we learn fixed length format and repeat. Study next example, which prints the special character usage list of the input file.

(2-11)

DfromLine "c:(1)*" websmpl.log | \
Dunbundle c | \
Dselect c UNLIKE "[ a-zA-Z0-9]" | \
Dfreq c | Dpr

The first DfromLine reads field c. The format (1) means that the field is fixed one character length, and ASTERISK (*) means the field repeats to the end of input line. The output from this command is like the following.

(2-12)

Output from the DfromLine of (2-11)

c:2 c:2 c:3 c:. c:9 c:8 ... ... c:0 c:5 c: c:4	-- 1st D-record (97 fields)
c:1 c:0 c:9 ... ... c: c:1	-- 2nd D-record (88 fields)
... ...
... c:6 c:9 c: c:2	-- 8th (last) D-record (84 fields)

This output has eight D-records corresponding to the number of lines in the input file. Each D-record has repeating field c, corresponding to each characters in the line.

Next Dunbundle disjoints the repeating field c, and generates 734 (total number of characters in the input lines) records having just one field c like the following.

(2-13)

Output from the Dunbundle of (2-11)

c:2 c:2 c:3 c:. c:9 c:8 ... ... c:9 c: c:2

-- 734 D-records,

Next Dselect command excludes SPACE, alphabets and digits. Dfreq counts the frequency of c. Final output of (2-11) will be the following.

(2-11 output)

rec# c count
   1 "    17
   2 +     8
   3 -    24
   4 .    32
   5 /    39
   6 :    24
   7 =     5
   8 ?     5
   9 [     8
  10 \     1
  11 ]     8

The second feature we learn is pattern. We have learned field specific delimiter can be given in a field-format-list with /regular expression/. Field pattern is, in a sense, the reversal of field delimiter. It has form @regular expression@, and gives the span of the field with the regular expression. We used delimiter /] / for the field time in (2-10). (Almost) same thing can be done with pattern. See the next command.

(2-14)

DfromLine -t " " \
"host,logname,user,time:+1@[^]]+@,request:+2qx,status,bytes,etime" \
websmpl.log

The field time, here, has pattern @[^]]+@ instead of delimiter /] /. This pattern reads as "one or more characters except RIGHT SQUARE BRACKET (])". The result is same as delimiter /]/. But, slight difference is found in the next field reading position. In the case of delimiter, next field starts from the character after the delimiter. But, in the case of pattern, next field starts from the character just after the matched pattern. It is the reason we need +2 for the next field request.

With pattern and repeat, we can pick up numbers from a text file.

(2-15)

DfromLine "num:@[0-9]+@*" websmpl.log

When a pattern does not match from the current character position, next field starts from the next character. The output has 8 D-records corresponding to the input lines. Each D-record has repeating field num of which value is a number appeares in the input line.

2.4 DfromChunk

DfromLine premises that a logical record is represented in a line. This type of data is commonly used in UNIX world. But, for example, ".ini" file format in Windows uses a text line to represent a logical field, and uses a group of lines to represent a logical record. This type of data representation is widely used (including D). E-mail header is one of the formats of this type. DfromChunk is general converter for this type of format.

2.5 D-file creation from NULL

This subsection use Ded without any input file. The following command produces partial ASCII character list.

(2-21)

Ded \
  FOR VAR i IN 0x60 .. 0x7e DO \
    code = VAR i ";" \
    hex = VAR i SPRINTF "%x" ";" \
    char = VAR i SPRINTF "%lc" ";" \
    OUTPUT \
  DONE \
< /dev/null  | \
Dpr

Note: Line division and indention are for readability. You may write these lines in one line.
< /dev/null should be < NUL in Windows command line.

Output will be:

(2-21 output)

rec# code hex char
   1   96 60  `
   2   97 61  a
   3   98 62  b
   4   99 63  c
   5  100 64  d
   6  101 65  e
   7  102 66  f
   8  103 67  g
   9  104 68  h
  10  105 69  i
  11  106 6a  j
  12  107 6b  k
  13  108 6c  l
  14  109 6d  m
  15  110 6e  n
  16  111 6f  o
  17  112 70  p
  18  113 71  q
  19  114 72  r
  20  115 73  s
  21  116 74  t
  22  117 75  u
  23  118 76  v
  24  119 77  w
  25  120 78  x
  26  121 79  y
  27  122 7a  z
  28  123 7b  {
  29  124 7c  |
  30  125 7d  }
  31  126 7e  ~

We have already used Ded in (1-38) and in (1-39). It is a general editor of D-files, and usually modifies input D-file with a Dl script. In this (2-21), we use Ded without any input. In fact, this script is run under epilogue mode of Ded.

Operator SPRINTF is like C function sprintf(). x SPRINTF y corresponds to sprintf(result, y, x). In this example, the first SPRINTF operation yields hexadecimal representation of VAR i. And, the second SPRINTF operation yields a long char with internal code value VAR i. You may use %c, but you can extend to non ASCII characters by using %lc.

Operator OUTPUT writes current field values as a D-record. At this timing, only field code, hex and char exist. (VAR i is not a field). So, the output has these three fields with current value. Note that automatic output at the end of cycle is not performed in epilogue mode

3.1 Simple repeat and Dunbundle

See the following example D-file movies.d.

(3-1)

movies.d

title:The Godfather
year:1972
director:Francis Ford Coppola
writer:Mario Puzo
writer:Francis Ford Coppola
star:Marlon Brando
star:Al Pacino

title:Singin' in the Rain
year:1952
director:Stanley Donen
director:Gene Kelly
star:Gene Kelly
star:Donald O'Connor
star:Debbie Reynolds

The records are about movies and a movie has one or more directors, writers, actors, etc. In SQL database (or Microsoft Access), repeating fields are not allowed. You have to separate tables like next example (assuming title and year provide the primary key).

(3-2)

Normalized tables

director table title year director The Godfather 1972 Francis Ford Coppola Singin' in the Rain 1952 Stanley Donen Singin' in the Rain 1952 Gene Kelly

writer table title year writer The Godfather 1972 Mario Puzo The Godfather 1972 Francis Ford Coppola

star table title year star The Godfather 1972 Marlon Brando The Godfather 1972 Al Pacino Singin' in the Rain 1952 Gene Kelly Singin' in the Rain 1952 Donald O'Connor Singin' in the Rain 1952 Debbie Reynolds

This is good for machine processing, but not for human reading. Repeating fields in the first D-file provides better browsability. When you are prparing the data by hand, the former formation is much easier.

D-file operations provide facilities to change the former formation to the latter (normalization), or the latter to the former.

(3-3)

Normalization process of movies.d

Dproj title,year,director movies.d | \
Dunbundle director > director.d

Dproj title,year,writer movies.d | \
Dunbundle writer > writer.d

Dproj title,year,star movies.d | \      output
Dunbundle star > star.d

These three groups of commands separates the D-file to three tables above. Procedures are all same. I'll take the third one (star.d) as an example.

The first Dproj limits the fields to be used ones. (See the output). The essential part is Dunbundle. It categorizes fields into "leaf" and "stem". "Leaf" field is specified by command parameter leaf-field-list, All other fields are "stem" fields. Dunbundle operation is to assume a tree structure between the "stem" fields and the "leaves", and to create N records each of which has the "stem" fields and an occurence of "leaf" fields, where N is the number of "leaves". The next figure shows this operation for the first record of the example.

Figure 3-1 Dunbundle operation

The leaf field is star. The stem fields are title and year. There are two "leaves" Marlon Brando and Al Pacino, hence the output has two records.

Dproj and Dunbundle operations above can be inverted. Dunbundle star movies.d will produce the following result.

(3-4)

title:The Godfather
year:1972
director:Francis Ford Coppola
writer:Mario Puzo
writer:Francis Ford Coppola
star:Marlon Brando

title:The Godfather
year:1972
director:Francis Ford Coppola
writer:Mario Puzo
writer:Francis Ford Coppola
star:Al Pacino

title:Singin' in the Rain
year:1952
director:Stanley Donen
director:Gene Kelly
star:Gene Kelly

title:Singin' in the Rain
year:1952
director:Stanley Donen
director:Gene Kelly
star:Donald O'Connor

title:Singin' in the Rain
year:1952
director:Stanley Donen
director:Gene Kelly
star:Debbie Reynolds

In this case, the leaf field is same star, but, the stem fields are title, year, director and writer, The fields director and writer actually repeats. But, they are just copied to the output records as they are, because they are stem fields. Only leaf fields are the target of unbundling.

It will be easyly understood that Dproj title,year,star to this output will produce same star.d.

3.2 Reverse operation of Dunbundle

It is quite natural to ask a question that "is Dbunbundle reversible operation?". The answer is not always "yes", but in many cases including the preceding example, it is reversible.

(3-5)

Reverse operation of Dunbundle (movies.d)

Dbundle title,year director.d > tmp-director.d
Dbundle title,year writer.d > tmp-writer.d
Dbundle title,year star.d > tmp-star.d
Djoin -c title,year tmp-director.d tmp-writer.d tmp-star.d

Dbundle command does the reverse operation of Dunbundle. Here, again, the third one (star.d) will be the example. The parameter title,year gives the stem fields. The fields of the input records are divided into stem fields and leaf fields as shown by the colors in the following.

(3-6)

star.d

title:The Godfather
year:1972
star:Marlon Brando

title:The Godfather
year:1972
star:Al Pacino

title:Singin' in the Rain
year:1952
star:Gene Kelly

title:Singin' in the Rain
year:1952
star:Donald O'Connor

title:Singin' in the Rain
year:1952
star:Debbie Reynolds

Then, Dbundle groups adjoining input records by the value of stem field, and merges the records in a group having:

• a set of stem fields and
• each leaf field of the record.

In the example above, there are two distinct stem field values: title:The Godfather; year:1972 and title:Singin' in the Rain; year:1952, so, the output has two records as follows:

(3-7)

tmp-star.d

title:The Godfather
year:1972
star:Marlon Brando
star:Al Pacino

title:Singin' in the Rain
year:1952
star:Gene Kelly
star:Donald O'Connor
star:Debbie Reynolds

The last part of the process Djoin do the reverse operation of three Dprojs in the "normalization" process (3-3). It takes two or more input files and merges the same key field (title,year here) value records from each input file. It recovers the original table form from divided three tables.

3.3 Pitfalls of this reverse operation (Advanced)

The method introduced in this section may not be successful under some circumstances. There are two pitfalls in the procedure. One is in Dbundle and the other is in Djoin. The common point is in the record sequence and the primary key.

3.4 Value of repeating fields

When a field repeats, you have to heed that the value is not single element. For example for movies.d,

(3-13)

Dselect star == "Marlon Brando" movies.d

does not select title:The Godfather record.

Because the value of repeating field is interpreted as an array of elements. With Dl expreession, the value of the field star is { "Marlon Brando" "Al Pacino" } which has two elements, apparently being different from single element "Marlon brando".

You may use INCL operator or more commonly used LIKE operator to test elements of an array.

When a repeating field is specified in the key-field-list of Dbundle, records are merged if the number of field occurences are same and all the value with their order are same. See next examples:

(3-14)

Repeating field equality by examples

Two field "a"s are equal record 1 record 2 a:1 a:1 a:2 b:2 a:3 a:2   c:3   a:3

Three field "a:n"s are equal record 1 record 2 record 3 a:1 a:01 a:1.0 a:2 a:02 a:2.0 a:3 a:03 a:3.0

Four field "a"s are unequal record 1 record 2 record 3 record 4 a:1 a:1 a:1 a:01 a:2 a:3 a:2 a:02 a:3 a:2   a:03

3.5 Repeating group

Here, the term "repeating group" refers to a set of fields which repeats as a group. For the example of repeating groups, we are going to use a record from Robert Schumann's work list.

(3-15)

schumann-op15.d

op:15
date:1838
title:Kinderscenen
category:keyboard
instrumentation:piano
pub:1839
part:1
ptitle:Von fremden Landern und Menschen
key:G
part:2
ptitle:Curiose Geschichte
key:D
part:3
ptitle:Hasche-Mann
key:h
part:4
ptitle:Bittendes Kind
key:D
part:5
ptitle:Glückes genug
key:D
part:6
ptitle:Wichtige Begebenheit
key:A
part:7
ptitle:Träumerei
key:F
part:8
ptitle:Am Camin
key:F
part:9
ptitle:Ritter vom Steckenpferd
key:C
part:10
ptitle:Fast zu ernst
key:gis
part:11
ptitle:Fürchtenmachen
key:G
part:12
ptitle:Kind im Einschlummern
key:e
part:13
ptitle:Der Dichter spricht
key:G

Well, this record may not be easy to read, but it is fairly simple. Schumann's op.15 "Kinderscenen" (Child-scenes) is made of thirteen piano pieces. While the field op (opus), date (year of composition), title, category, instrumentation and pub (year of publishing) describe the whole work, the field part (sequence number of the piece), ptitle (title of the piece) and key describe each piece, so that the latter fields repeat thirteen times. We can think former fiedlds form a stem and the latter fields form thirteen leaves. Next command:

(3-16)

Dtie -t ", " part,ptitle,key piece schumann-op15.d

produces the following output.

(3-16 output)

op:15
date:1838
title:Kinderscenen
category:keyboard
instrumentation:piano
pub:1839
piece:1, Von fremden Landern und Menschen, G
piece:2, Curiose Geschichte, D
piece:3, Hasche-Mann, h
piece:4, Bittendes Kind, D
piece:5, Glückes genug, D
piece:6, Wichtige Begebenheit, A
piece:7, Träumerei, F
piece:8, Am Camin, F
piece:9, Ritter vom Steckenpferd, C
piece:10, Fast zu ernst, gis
piece:11, Fürchtenmachen, G
piece:12, Kind im Einschlummern, e
piece:13, Der Dichter spricht, G

This record is better for reading. Dtie finds tree structure in the input record, like Dunbundle, and "ties" leaf fields making individual leaf as a new field. In other words, it converts a repeating group into a single repeating field. In the command line, part,ptitle,key is the leaf field list, piece is the new field name given to the "tied" leaf, and the option -t ", " gives the delimiter to be inserted between "tied" values.

Here, you may ask question that how Dtie find the leaves. By default, Dtie uses N-th occurence algorithm. It makes n-th leaf from n-th occurence of each leaf field. But, this algorithm does not work well in some cases. See another record from Robert Schumann's work list.

(3-17)

schumann-op14a.d

op:14a
date:1836
title:Sonate No.3
atitle:Grosse Sonate
atitle:Concert ohne Orchester
version:Erste Ausgabe
category:keyboard
instrumentation:piano
pub:1836
part:1
tempo:Allegro brillante
key:f
part:2
ptitle:Quasi variazioni
tempo:Andantino de Clara Wieck
key:f
part:3
tempo:Prestissimo possibile
key:f

There are three new fields: atitle (alternative title: repeating) and version pertain to the whole work and tempo pertains to each piece ("movement"). If you use

(3-18)

Dtie -t ", " part,ptitle,tempo,key movement schumann-op14a.d

it will produce the following output.

(3-18 output)

op:14a
date:1836
title:Sonate No.3
atitle:Grosse Sonate
atitle:Concert ohne Orchester
version:Erste Ausgabe
category:keyboard
instrumentation:piano
pub:1836
movement:1, Allegro brillante, f, Quasi variazioni
movement:2, Andantino de Clara Wieck, f
movement:3, Prestissimo possibile, f

This is not good because "Quasi variazioni" should come to the second movement. With N-th occurence algorithm, missing fields in leaves are not allowed. In this op.14a record (and in other records), description of pieces are always led by part field. In such case Leading field algorithm works well. Use following command:

(3-19)

Dtie -t ", " -L part,ptitle,tempo,key movement schumann-op14a.d

will produce the next output.

(3-19 output)

op:14a
date:1836
title:Sonate No.3
atitle:Grosse Sonate
atitle:Concert ohne Orchester
version:Erste Ausgabe
category:keyboard
instrumentation:piano
pub:1836
movement:1, Allegro brillante, f
movement:2, Quasi variazioni, Andantino de Clara Wieck, f
movement:3, Prestissimo possibile, f

Note that Leading field algorithm works for op.15 as well. For the detail of these "leaf separation algorithims", refer to the manual D_lsa.

3.6 Repeating group and Dfd

We used Dfd command in the "Getting started" section (1-3). Before going to the next session. we observe the result of Dfd command on the repeating groups. Do next command.

(3-20)

Dcat schumann-op14a.d schumann-op15.d > schumann-ops.d

It produces a D-file with two records we used in the previous subsection. Try Dfdp command.

(3-21a)

Dfdp schumann-ops.d

or

(3-21b)

Dfd schumann-ops.d | Dpr -f exists:r -g filename

It will produce the following output.

(3-21 output)

                            filename:schumann-ops.d

rec# fieldname       min max exists minlen maxlen avglen  attribute position
   1 op                1   1    2/2      2      3  2.5    Asc
   2 date              1   1    2/2      4      4  4      Int
   3 title             1   1    2/2     11     12 11.5    Asc
   4 atitle            0   2    1/2     13     22 17.5    Asc
   5 version           0   1    1/2     13     13 13      Asc
   6 category          1   1    2/2      8      8  8      Asc
   7 instrumentation   1   1    2/2      5      5  5      Asc
   8 pub               1   1    2/2      4      4  4      Int
   9 part              3  13    2/2      1      2  1.25   Int       split
  10 ptitle            1  13    2/2      8     32 16.5    AscNAsc   split
  11 tempo             0   3    1/2     17     24 20.6667 Asc       split
  12 key               3  13    2/2      1      3  1.125  Asc       split

The output has a new column named "position", which is not found in the (1-3 output). The value of this field is either null or split. This value means that the field is repeated and there are other fields among the occurecnces of this field (in at least one record). This split strongly suggests the field is a part of a repeating group.

In this case, fields part, ptitle, tempo and key have value split and likely to form a repeating group.

Note that the field ptitle has value AscNAsc. Some of the part titles have German words with umlauts which can not be represented by ASCII characters.

3.7 Djoin, Dbundle and repeating group

As we already saw in "D-file operations" section, repeating group is produced with D-file operation Dbundle from normalized files. We are going back to Supplier-part-project (spj) example for this section.

In the spj files, all the essential associations between three entities (supplier, part and project) are represented in supply.d file. Suppose you are the project manager of the "Console" project (j# == 4), you will need to know what kind of parts you use, and which supplier(s) supply them when and how many. You may use

(3-22)

Dsort j#,p#,s# supply.d | Dbundle j#,p# | Dselect j# == 4 | Dpr

to get the followig result.

(3-22 output)

rec# s# p# shipdate quantity
   1  1  1 70,1,12       700
      5    70,2,9        100
   2  5  2 70,1,23       100
   3  2  3 70,1,17       500
      5    70,1,14       200
   4  5  4 70,1,24       800
   5  5  5 70,1,12       400
   6  5  6 70,2,5        500

Dbundle j#,p# makes tree structure with repeating groups. Stem fields are j#,p# and leaf fields are p#,shipdate,quantity. The first Dsort j#:n,p#:n,s#:n prepares for the Dbundle.

If you memorize who is supplier number 5 and what is part number 2, this result may be ok. But, general people need part names and supplier names in addition to p# and s#. You can find these names in part.d or in supplier.d. For the part names, the following command

(3-23)

Dsort j#,p#,s# supply.d | Dbundle j#,p# | Dselect j# == 4 | \
Djoin -c p# - part.d | \
Dorder j#,p#,pname,color,weight,qoh,s#,shipdate,quantity | Dpr

produces next output.

(3-23 output)

rec# j# p# pname color weight qoh s# shipdate quantity
   1  4  1 Nut   Red       12  30  1 70,1,12       700
                                   5 70,2,9        100
   2  4  2 Bolt  Green     17  60  5 70,1,23       100
   3  4  3 Screw Blue      17  30  2 70,1,17       500
                                   5 70,1,14       200
   4  4  4 Screw Red       14  10  5 70,1,24       800
   5  4  5 Cam   Blue      12  20  5 70,1,12       400
   6  4  6 Cog   Red       19  40  5 70,2,5        500

Dorder here is just to provide easier reading.

But, you can not combine supplier names in the same way. The field p# is one of the stem fields, while the field s# is one of the leaf fields in the (3-22 output). You do not join files with a leaf field.

If you try the same way for the field s# and supplier.d as follows,

(3-24)

Dsort j#,p#,s# supply.d | Dbundle j#,p# | Dselect j# == 4 | \
Djoin -c p# - part.d | \
Djoin -c s# - supplier.d | \
Dorder j#,p#,pname,color,weight,qoh,s#,sname,loc,shipdate,quantity | Dpr

You will get

(3-24 output)

rec# j# p# pname color weight qoh s# sname loc    shipdate quantity
   1  4  1 Nut   Red       12  30  1              70,1,12       700
                                   5              70,2,9        100
   2  4  2 Bolt  Green     17  60  5 Adams Athens 70,1,23       100
   3  4  3 Screw Blue      17  30  2              70,1,17       500
                                   5              70,1,14       200
   4  4  4 Screw Red       14  10  5 Adams Athens 70,1,24       800
   5  4  5 Cam   Blue      12  20  5 Adams Athens 70,1,12       400
   6  4  6 Cog   Red       19  40  5 Adams Athens 70,2,5        500

Some of the supplier names and locations are missing. The reason is almost obvious if you understand the Value of repeating fields subsection. As the field s# is repeating, it does not match with the records in supplier.d, which has single value s#.

Solution is, again, almost obvious. Do Djoin before Dbundle.

(3-25)

Dsort j#,p#,s# supply.d | \
Djoin -c p# - part.d | \
Djoin -c s# - supplier.d | \
Dbundle j#,p#,pname,color,weight,qoh | \
Dselect j# == 4 | Dorder j#,p#,pname,color,weight,qoh,s#,sname,loc,shipdate,quantity | Dpr

This is almost same as (3-24) but the Dbundle comes after two Djoins. In addition, note that the parameter of Dbundle includes all the dependent fields of p# (sname,color,weight,qoh). This command will produce output like this.

It is easy to modify (3-25) so that not only the number 4 project but all the projects are listed. For this, usually, -g (group by) option is used.

(3-26)

Djoin -c j# supply.d project.d | \
Djoin -c p# - part.d | \
Djoin -c s# - supplier.d | \
Dsort j#:n,p#:n,s#:n | \
Drder j#,jname,mgr_e#,p#,pname,color,weight,qoh,s#,sname,loc,shipdate,quantity | \
Dbundle j#,jname,mgr_e#,p#,pname,color,weigt,qoh | \
Dpr -g j#,jname,mgr_e#

This command produces output like this.

3.8 A bit theoretically

Before starting this topic, we introduce the consolidated file spj.d for Spj examples as follows.

(3-27)

Djoin -c s# supply.d supplier.d | \
Djoin -c p# - part.d | \
Djoin -c j# - project.d > spj.d

You know original supplier.d, part.d, project.d and supply.d files can be easily restored from spj.d file. For example:

(3-28)

Dproj s#,sname,loc spj.d | \
Dsort s#:n,sname,loc | Dbundle "^" > supplier.d

Now, Spj examples can be viewd as three level tree structure with six ways, They are:

• supplier - part - project
• supplier - project - part
• part - supplier - project
• part - project - supplier
• project - supplier - part
• project - part - supplier

The output of (3-26) is project - part - supplier tree, Actually, the top level project has no corresponding record. It is represented only by -g j#,jname,mgr_e# option of the Dpr command, while part and supplier level is represented by the stem - leaf structure in each D-record.

In the same way, you can make supplier - part - project tree with the following command.

(3-29)

Dsort s#:n,p#:n,j#:n spj.d | \
Dorder s#,sname,loc,p#,pname,color,weight,qoh,j#,jname,mgr_e#,shipdate,quantity | \
Dbundle s#,sname,loc,p#,pname,color,weight,qoh | \
Dpr -g s#,sname,loc

It will produce an output like this.

Is it possible to make three level tree structure? Yes. The following commands produce supplier - part - project tree.

(3-30)

Dsort s#:n,p#:n,j#:n spj.d | \
Dorder s#,sname,loc,p#,pname,color,weight,qoh,j#,jname,mgr_e#,shipdate,quantity | \
Dbundle s#,sname,loc,p#,pname,color,weight,qoh | \
Dbundle s#,sname,loc > 3-level.d

Fields s#, sname and loc are the first level stem fields. Fields p#, pname, color, weight and qoh are the second level, i.e. leaf for the first level and stem for the third level. Fields j#, jname, mgr_e#, shipdate and quantity are the third level leaf fields. In the listing of 3-level.d, these are shown by color.

If you want to draw out the second level,

(3-31)

Dunbundle -L p#,pname,color,weight,qoh,j#,jname,mgr_e#,shipdate,quantity 3-level.d

will work. Note that you need -L option (Leading Field Algorithm). Note also that you need to list the third level leaf fields, too.

If you want to draw out the third level, you need Dunbundle after extracting the second level (3-31).

(3-32)

Dunbundle -L p#,pname,color,weight,qoh,j#,jname,mgr_e#,shipdate,quantity 3-level.d | \
Dunbundle -L j#,jname,mgr_e#,shipdate,quantity

As shown in this example, three level tree structure can be embedded in the D-record. But, D-commands support, basically, two level tree structure only. You need to iterate D-commands to process lower level leaves. For this reason, it is not good idea to embed three or more level tree structure into a D-file.

3.9 Conversion to XML

When handling three or more level tree structure, explicitly, you should consider to convert the data to XML, as D is basically for flat or two level tree structure.

DtoXml is the command to use. (There is, of course, DfromXml and more popular DfromHtml. They are introduced in a separate section.)

Suppose you have the following file ab.d.

(3-33)

a:123
b:456

Then, the command

(3-34)

DtoXml ab.d

produces the following output.

(3-35)

<root>
 <file>
  <record>
   <a>123</a>
   <b>456</b>
  </record>
 </file>
</root>

(Note that unnecessary spaces are removed and indention is added for readability.)

This is for flat files. DtoXml offers two ways of giving structure. One way is giving leaf field, and the other way is using group by (-g) option. Leaf fields can represent two level tree structure. Group by options can be used more than one, so that arbitrary number of levels can be represented. In addition, group by and leaf fields can be used together.

To show the usage of DtoXml, project - part - supplier tree made by (3-26) and supplier - part - project tree made by (3-30) will be produced with different way.

For the first way, Dsort, Dorder (for the readability) and Dbundle are same as we did in (3-26). In the last part, we use DtoXml instead of Dpr

(3-36)

Dsort j#:n,p#:n,s#:n | \
Drder j#,jname,mgr_e#,p#,pname,color,weight,qoh,s#,sname,loc,shipdate,quantity | \
Dbundle j#,jname,mgr_e#,p#,pname,color,weigt,qoh | \
Drename j#:jno,mgr_e#:mgr_eno,p#:pno,s#:sno | \
DtoXml -g jno,jname,mgr_eno -L sno,sname,loc,shipdate,quantity

Drename is to change field names not allowed for XML tag names. The output is listed here.

The other way is more simple. It starts just like (3-30) and instead of Dbundle, DtoXml with two group by options is used.

(3-37)

Dsort s#:n,p#:n,j#:n spj.d | \
Dorder s#,sname,loc,p#,pname,color,weight,qoh,j#,jname,mgr_e#,shipdate,quantity | \
Drename s#:sno,p#:pno,j#:jno,mgr_e#:mgr_eno | \
DtoXml -g sno,sname,loc -g pno,pname,color,weight,qoh

The output will be this. Though the tag names are slightly different, those two are essentially same XML document. If you want, you can change tag names like <group>, <record> or <leaf>. See the manual of DtoXml.

How to handle XML files is out of scope of this tutorial. But, before closing this section, an HTML tables produced from the output of (3-37) are shown.

(3-38)

Table 3-1. An example of tables produced from the XML file

j#:1; Smith, London

p#	pname	color	weight	qoh	j#	jname	mgr_e#	shipdate	quantity
1	Nut	Red	12	30	1	Sorter	640028	70,1,12	200
					4	Console	660118	70,1,12	700

j#:2; Jones, Paris

p#	pname	color	weight	qoh	j#	jname	mgr_e#	shipdate	quantity
3	Screw	Blue	17	30	1	Sorter	640028	70,1,17	400
					2	Punch	700128	70,1,17	200
					3	Reader	680096	70,1,17	200
					4	Console	660118	70,1,17	500
					5	Collator	690002	70,1,17	600
					6	Terminal	640014	70,1,17	400
					7	tape	710020	70,1,17	800
5	Cam	Blue	12	20	2	Punch	700128	69,12,19	100

j#:3; Blake, Paris

p#	pname	color	weight	qoh	j#	jname	mgr_e#	shipdate	quantity
3	Screw	Blue	17	30	1	Sorter	640028	70,2,2	200
4	Screw	Red	14	10	2	Punch	700128	70,2,10	500

j#:4; Clark, London

p#	pname	color	weight	qoh	j#	jname	mgr_e#	shipdate	quantity
6	Cog	Red	19	40	3	Reader	680096	70,1,31	300
					7	tape	710020	70,1,31	300

j#:5; Adams, Athens

p#	pname	color	weight	qoh	j#	jname	mgr_e#	shipdate	quantity
1	Nut	Red	12	30	4	Console	660118	70,2,9	100
2	Bolt	Green	17	60	2	Punch	700128	70,1,23	200
					4	Console	660118	70,1,23	100
3	Screw	Blue	17	30	4	Console	660118	70,1,14	200
					7	tape	710020	70,1,31	100
4	Screw	Red	14	10	4	Console	660118	70,1,24	800
5	Cam	Blue	12	20	4	Console	660118	70,1,12	400
					5	Collator	690002	70,1,12	500
					7	tape	710020	70,1,12	100
6	Cog	Red	19	40	2	Punch	700128	70,1,16	200
					4	Console	660118	70,2,5	500

HTML source here

These HTML tables are produced from the output of (3-37) with this xslt script.

4.1 Simply Drename

Of course, Drename provides field rename facility. For example,

(4-2)

Drename cc:country-code,nm:name,lc:language-code rename-1.d              Output

renames field cc to country-code, field nm to name and field lc to language-code. (It's good idea to make expressive field names for the readability). But, what if Marianne says it's not name but nom, and Michel says it's Name?

4.2 Separation and Drename

Drename does not provide conditional renaming function. The first idea is to divide the data by language and apply Drename separately.

(4-3)

Dselect lc == de rename-1.d | Drename nm:Name > tmp-de.d
Dselect lc == fr rename-1.d | Drename nm:nom > tmp-fr.d
Dselect lc == en rename-1.d | Drename nm:name > tmp-en.d
Dcat tmp-de.d tmp-fr.d tmp-en.d                                     Output

Note: lc == de is FIELD lc == CONST de in full style. It may be better to use the full style to avoid unexpected result.

But, with this method you needs more than three paths, i.e., you have to read through whole file more than three times. This is not efficient when the file is very big or you have large number of conditions you want to separate. Isn't there a method with just one path?

4.3 Using Ded

Ded is an interpreter of Dl language and a general purpose D-record processor.

(4-4)

Ded IF lc == de THEN FIELD Name = FIELD nm \
    ELIF lc == fr THEN FIELD nom = FIELD nm \
    ELSE FIELD name = FIELD nm FI ";" \
    FIELD nm = { } rename-1.d                      Output

What this small Dl program does is to make a new field depending on the lc value, copying from nm field, and then remove nm field ( nm = { } ). As the new field is generated after the existing fields, the field order is changed. If the field order matters, you may use Dorder after this.

4.4 Tricky but effective

This method also uses Dl with Ded. It is tricky because it relies on the raw structure of D-record.

(4-5)

Ded IF lc == de THEN @_ = @_ S "^nm:" BY "Name:" \
    ELIF lc == fr THEN @_ = @_ S "^nm: BY "nom:" \
    ELSE @_ = @_ S "^nm:" BY "name:" FI rename-1.d              Output

Note: >@_ can be FIELDS or CURREC. These three are synonyms.

This Dl program is similar to the previous one. The point is @_ and S operator. @_ is the current record special variable, which is an array of all fields in the current record represented as strings in the form of D-fields in a D-file (i.e., field name:value form). It can also be the left hand value of an asignment operator, and changes made in @_ is directly reflected in the current record.

S operator is ternary substitute operator: x S y BY z means "s/y/z/" operation of sed on the string x. The second operand y is regular expression. In this expression, ^nm: matchs the string nm: at the beginning of the string, hence matches the field name nm. In this case, @_ is an array. S operation is applied to each elements of @_, and the result is same size array.

5.1 Using Ddecompose and Dcompose

Sometimes, you may want to transpose rows and columns in a table. In D-files, this means to convert a field value to a field name, and to convert field names to field values under some field name. For example, for a D-file transposition-1.d shown with Dpr -i year transposition-1.d as:

(5-1)

          transposition-1.d

year USD/CHF USD/EUR USD/GBP USD/JPY
2001  1.6868  1.1166  0.6942 121.5193
2002  1.5561  1.0608  0.6663 125.1998
2003  1.3449  0.8852  0.6122 115.9517
2004  1.2426  0.8050  0.5460 108.1542
2005  1.2455  0.8044  0.5500 110.1053 Text view here.

You may want to transpose this table into row as time series form, such as:

(5-2)

        transposition-2.d

currency 2001     2002     2003     2004     2005
USD/CHF    1.6868   1.5561   1.3449   1.2426   1.2455
USD/EUR    1.1166   1.0608   0.8852   0.8050   0.8044
USD/GPB    0.6942   0.6663   0.6122   0.5460   0.5500
USD/JPY  121.5193 125.1998 115.9517 108.1542 110.1053 Text view here

There is no direct D-operation to do transposition. Use Ddecompose to make triplets of row-id, column-id and the value (e.g, 2001, USD/CHF, 1.6868), then Dsort to make it in column order, and finally use Dcompose to assemble in new rows.

(5-3)

Ddecompose -i year transposition-1.d | \           Output, Note 1
Dsort fieldname | \                                Output
Drename year:fieldname,fieldname:currency | \      Output, Note 2
Dcompose -i currency > transposition-2.d

Note 1: year is the row identifier field in transposition-1.d file.
Note 2: currency is the field name given here for the new row identifier field.

5.2 Tricky and unrecommendable way (for backward compatibility)

There is another way to do transposition. This is a bit tricky and not recommendable, but as Ddecompose and Dcompose was introduced from D-2.6, you have to use this in the earlier versions of D. This procedure essentially does same process as the above Ddecompose-Dcompose procedure.

(5-4)

Dtie -zf "^year" .x transposition-1.d | \    Output, Note 3
Dunpack .x | \                               Output
Dunbundle .x | \                             Output
Duntie -t: .x currency,.y | \                Output, Note 4
Dsort currency | \                           Output
Dtie -t: year,.y .z | \                      Output, Note 5
Duntie .z .:f | \                            Output, Note 6
Dbundle currency | \                         Output
Dorder currency > transposition-2.d                 Note 7

Note 3: .x here is an arbitrary field name, local to this procedure. Dtie "^year" .x in this case ties all fields except for year with TAB character, forming a new .x field. In doing so with format option -zf, not only the value of the field, but also field name is tied in <field name>:<value> form. As it uses TAB as the delimiter, when your data has TAB character in it, you have to give different delimiter with -t option, and have to give the same delimiter in the next Dunpack command.
Note 4 currency is the field name given here, .y is an arbitrary field name, local to this procedure. Note that row-id, column-id, value triplet is made here.
Note 5: year field and the value field (.y) is tied with delimiter COLON and temporary field name .z is given.
Note 6: Duntie here is used just to read the contents of .z field. AS -zf format is given, actual field name is derived from the contents of .z field.
Note 7 This Dorder is not essential, but at this position, the first field is 2001. It is not same as transposition-2.d. This Dorder command makes field order as in transposition-2.d.

6.1 Simple table reading

We use the following HTML table In this section.

(6-1)

HTML-table-1

Country-name	Country-code	Language-code	Currency-code
FRANCE	FRA	fr	EUR
GERMANY	DEU	de	EUR
JAPAN	JPN	ja	JPY
KOREA, REPUBLIC OF	KOR	ko	KRW
UNITED KINGDOM	GBR	en	GBP

HTML source here

6.2 Reading a table with a header column

We use the following HTML table In this section.

(6-6)

HTML-table-2

	HIGH	LOW
03-25	12°C	2°C
03-26	11°C	1°C
03-27	10°C	1°C
03-28	10°C	2°C

HTML source here

6.3 Table with colspan, rowspan

We use the following HTML table In this section.

(6-10)

HTML-table-3

Year	Month	Tokyo		Sapporo
		Maximum Temperature	Minimum Temperature	Maximum Temperature	Minimum Temperature
2004	Oct	28°C	9°C	24°C	0°C
	Nov	23°C	8°C	19°C	-5°C
	Dec	25°C	0°C	11°C	-9°C
2005	Jan	19°C	-1°C	7°C	-12°C
	Feb	19°C	0°C	3°C	-11°C
	Mar	19°C	1°C	9°C	-11°C

HTML source

Unlike former two tables, how to read this table as a D-file is not straightforward. You may want to make records like

(6-11)

Year:2004
Month:Oct
Tokyo Maximum Temperature:28°C
Tokyo Minimum Temperature:9°C
Sapporo Maximum Temperature:24°C
Sapporo Minimum Temperature:0°C

or, like

(6-12)

Year:2004
Month:Oct
Place:Tokyo
property:Maximum Temperature
value:28°C

Year:2004
Month:Oct
Place:Tokyo
property:Minimum Temperature
value:9°C

There are many other ways. It depends on your purpose to choose the form of the result. Here, two examples above will be shown.

6.4 Locating your table, or How to write xpath parameter

In the previous examples, we handled only one table in an HTML document. We used only -p //tr, for the DfromHtml, and it worked well. But, real HTML documents often have more than one tables in a document. In these cases, we have to provide xpath expression beyond simple //tr with the -p option.

7.1 Dhead and Dtail

Dhead is rarely used D-commnad, because it can be substituted by Dextract. Dtail is a little more used, because you can extract last N records even in case you don't know the total number of records.

Next example extracts from the third last to the second last D-record.

(7-1)

Dtail -3 input-file | Dhead -2

7.2 DtoCsv

DtoCsv, naturally, converts D-files into a CSV file. It is a reverse conversion of DfromCsv mentioned in 1.3.

In many cases, DtoLine is enough to hand D-records to other applications. The advantage of DtoCsv is mostly in automatic header line creation.

Note that specification of "CSV" file is not standardized universally. Interpretation depends on the application. Csv/tsv files produced by this program, usually, are accepted by major applications. But, there may be some cases, they are not properly converted.

The following command produces a "TSV" file of supply.d

(7-2)

DtoCsv -t "TAB" supply.d

You may need special key sequence to input TAB control character. See paragraphs after (6-16).

7.3 DfromXml

DfromXml is almost same program as DfromHtml. As XML is better structured than HTML and many tools are available, it is not so often that you need to use DfromXml.

The most prominent difference between DfromXml from DfromHtml is namespace. The manual describes the detail. Usually, you use the default prefix D: and/or the implicit binding.

The first example is about the default namespace. This tutorial itself is written in XHTML, which is an XML document with the default namespace as HTML (http://www.w3.org/1999/xhtml). The listing of <h2> elements under the <body><div> level can be obtained by:

(7-3)

DfromHtml -p /html/body/div/h2 D_tutorial.html | Dpr

The output will be:

(7-3 output)

rec# node                  text
   1 /html/body/div[1]/h2  CONTENTS
   2 /html/body/div[2]/h2  1. Getting started
   3 /html/body/div[3]/h2  2. Creation and validation of D-files
   4 /html/body/div[4]/h2  3. When a field repeats
   5 /html/body/div[5]/h2  4. Renaming the fields
   6 /html/body/div[6]/h2  5. Row-column transposition
   7 /html/body/div[7]/h2  6. HTML table reading
   8 /html/body/div[8]/h2  7. Miscellaneous topics
   9 /html/body/div[9]/h2  Appendix A. D-command index
  10 /html/body/div[10]/h2 Appendix B: Data views

Note: This output is from a certain version of this tutorial, and man not be same as the output from current version.

The document is also an XML document. You may think the command (7-3) works just by changing DfromHtml to DfromXml. But, it will fail.

In the XHTML document, <html> tag has default name space declaration like <html xmlns="http://www.w3.org/1999/xhtml" >. DfromXml gives prefix D: for the default name space. You can use this prefix with (7-3).

(7-4)

DfromXml -p /D:html/D:body/D:div/D:h2 D_tutorial.html | Dpr

The output will be the following.

(7-4 output)

rec# node               text
   1 /*/*[2]/*[4]/*[1]  CONTENTS
   2 /*/*[2]/*[5]/*[1]  1. Getting started
   3 /*/*[2]/*[6]/*[2]  2. Creation and validation of D-files
   4 /*/*[2]/*[7]/*[2]  3. When a field repeats
   5 /*/*[2]/*[8]/*[2]  4. Renaming the fields
   6 /*/*[2]/*[9]/*[2]  5. Row-column transposition
   7 /*/*[2]/*[10]/*[2] 6. HTML table reading
   8 /*/*[2]/*[11]/*[1] 7. Miscellaneous topics
   9 /*/*[2]/*[12]/*[2] Appendix A. D-command index
  10 /*/*[2]/*[13]/*[2] Appendix B: Data views

In this output, node value has ASTERISK (*) instead of tag names. When name spaces are used, libxml2 processor returns these form node names, because it does not know what prefix is used in the external world (in this case DfromXml).

The second example -- implicit binding is more simple. See the next XML document, in a file document.xml, which is the main part of ".docx" document.

(7-5)

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<w:document
  xmlns:wp="http://schemas.openxmlformats.org/drawingml/2006/wordprocessingDrawing"
  xmlns:w10="urn:schemas-microsoft-com:office:word"
  xmlns:w="http://schemas.openxmlformats.org/wordprocessingml/2006/main"
  xmlns:v="urn:schemas-microsoft-com:vml"
  xmlns:r="http://schemas.openxmlformats.org/officeDocument/2006/relationships"
  xmlns:o="urn:schemas-microsoft-com:office:office">
  <w:body>
    <w:p>
      <w:pPr>
        <w:pStyle w:val="style0"/>
      </w:pPr>
      <w:r>
        <w:rPr/>
        <w:t>Test</w:t>
      </w:r>
    </w:p>
     
  </w:body>
</w:document>

There are many name space declaration, but actually used is only w: prefix. You can use this prefix w: to extract actual text.

(7-6)

DfromXml -p //w:t document.xml

The output will be the following.

(7-6 output)

node:/w:document/w:body/w:p/w:r/w:t
text:Test

Naturally, default name space prefix D: and inplicit name space binding has some limitations. See the manual for the detail.

7.4 DtoTex

TeX is being out of fassion. It is still useful in some fields. DtoTex provides a way to typeset data through TeX using TeX macro mechanism.

By default, D-field name is used as a TeX macro name. You have to write TeX macro definition correspoinding to the D-fields. See the next command.

(7-7)

DtoTex schumann-op14a.d

The output will be thw following.

(7-7 output)

\op 14a
\date 1836
\title Sonate No.3
\atitle Grosse Sonate
\atitle Concert ohne Orchester
\version Erste Ausgabe
\category keyboard
\instrumentation piano
\pub 1836
\part 1
\tempo Allegro brillante
\key f
\part 2
\ptitle Quasi variazioni
\tempo Andantino de Clara Wieck
\key f
\part 3
\tempo Prestissimo possibile
\key f

TeX special characters in the values are escaped automatially, or you may keep them with -n field list. You need your macro definition to typeset this output. It is beyond the scope of this document.

7.5 Making D-command index (Appendix A)

The following procedure produces the Annex A (D-command index) from the HTML file of this document. The actual procedure the author used is slightly different from (7-8), but essential procedures are same. This procedure depends greatly on the conventions the author used in this document, (e.g., commmand examples are blockquoted and example numbers are written with <em> tag with "expno" class), these commands, nevertheless, show many useful examples.

(7-8)

① DfromHtml -p //blockquote/p/em[@class=\"expno\"]/../.. D_tutorial.html  | \
② Ded IF EXISTS blockquote THEN FIELD blockquote = { } ';'  \
    @id [ 1 ] = { } FI ';'  \
    @id = CONST "#" . FIELD @id ';'  \
    expno = FIELD p [ 0 ] ';'  \
    FIELD p [ 0 ] = { }  | \
③ Dproj @id,expno,p  | \
④ Dunbundle p  | \
⑤ Dselect p LIKE "^D" AND @id UNLIKE CONST _  | \
⑥ Duntie p "command:@^D[a-z]+[A-Za-z]*@*"  | \
⑦ Dunbundle command  | \
⑧ Dsort command  | \
⑨ Dbundle command,@id,expno  | \
⑩ Dorder command,@id,expno  | \
⑪ Dbundle command  | \
⑫ Dtie '@id://:<a href="%ls">,expno::%ls</a>' td  | \
⑬ Dpack -t " " td  | \
⑭ Drename command:td  | \
⑮ DtoXml -o table -f tbody -r tr -n td

① The first DfromHtml selects <em> nodes with the attribute class="expno" under a <p> element under a <blockquote> element, and makes D-records with its grandparent (thus <blockquote>) node. Note that REVERSE SOLIDUS (\) is required to pass QUOTATION MARK (") to Xpath.

② does some preparative processes. If the D-record has blockquote field, that means the second level <blockquote> is used in it. By convention they are notes within the example and should be eliminated with the @id associated to the note. Then, @id should be only one. NUMBER SIGN (#) is added to @id to use the value as a hyper link. The first paragraph is, by convention, the example number. Rename it as expno, and remove itself from paragraphs.

③ removes fields not used in the following processes.

At this point, the D-records may have more than one paragraphs (p fields) as in (3-5). ④ unbundles them and makes one paragraph per one D-record.

⑤ excludes unnecessary records. D-command example paragraph starts always with a LATIN CAPITAL LETTER D. In addition, examples like (1-2 output) are eliminated.

⑥ extracts D-command names from the paragraph. @STRING@ format with Repeat (*) option can extract words mathced to the pattern from a line.

⑦ unbundles command field, so that the result record is a triplet having @id, command and expno.

⑧ and ⑨ is "uniq" operation to eliminate duplication. The result of ⑦ has duplications if the same command is used more than one time in a same example.

From this point we start to construct HTML table output by ⑮ DtoXml. ⑩ is to change the field order for the convenience of the following procedures.

The output record of ⑪ has a command field and one or more pairs of @id and expno field. The sequence of these leaf pairs reserves the sequence in the input file, Dsort in ⑧ keeps the input record order, unless specified by the sort key specification.

⑫ makes a field td from a pair of @id and expno field. The result value is, for example, <a href="#n-1-1">(1-1)</a>

⑬ makes repeating td fields into one. ⑭ renames the field command to td, so that a D-record has only two td fields, of which first one has the command name as the value, and the second one has HTML formatted example number list.

The last step ⑮ constructs HTML <table> tags using one D-record as a <tr>. Note that td field must be listed with -n option, because it is already in HTML representation.

B.1.1 supply.d (1-10) in 1.4.1

            supply.d

rec# s# p# j# shipdate quantity
   1  1  1  1 70,1,12       200
   2  1  1  4 70,1,12       700
   3  2  3  1 70,1,17       400
   4  2  3  2 70,1,17       200
   5  2  3  3 70,1,17       200
   6  2  3  4 70,1,17       500
   7  2  3  5 70,1,17       600
   8  2  3  6 70,1,17       400
   9  2  3  7 70,1,17       800
  10  2  5  2 69,12,19      100
  11  3  3  1 70,2,2        200
  12  3  4  2 70,2,10       500
  13  4  6  3 70,1,31       300
  14  4  6  7 70,1,31       300
  15  5  2  2 70,1,23       200
  16  5  2  4 70,1,23       100
  17  5  3  7 70,1,31       100
  18  5  5  5 70,1,12       500
  19  5  5  7 70,1,12       100
  20  5  6  2 70,1,16       200
  21  5  1  4 70,2,9        100
  22  5  3  4 70,1,14       200
  23  5  4  4 70,1,24       800
  24  5  5  4 70,1,12       400
  25  5  6  4 70,2,5        500

B.2.1 supply-handmade.d in 2.1

s#:1
p#:1
j#:1
shipdate:70,1,12
quantity:200

s#:1
p#:1
j#:4
shipdate:70,1,12
quantity:700

s#:2
p#:3
j#:1
shipdate:70,1,17
quantity:400

s#:2
p#:3
j#:2
shipdata:70,1,17
quantity:200

s#:2
p#:3
j#:3
shipdate:70,1,17
quantity:200

s#:2
p#:3
j#:4
shipdate:70,1,17
quantity:500

s#:2
p#:3
j#:5
shipdate:70,1,17
quantity:600

s#:2
p#:3
j#:6
shipdate:70,1,17
quantity:400

s#:2
p#:3
j#:7
shipdate:70,1,17
quantity:800

s#:2
p#:5
j#:2
shipdate:69,12,19
quantity:1.3

s#:3
p#:3
j#:1
shipdate:70,2,2
quantity:200

s#:3
p#:4
j#:2
shipdate:70,2,10
quantity:500

s#:4
p#:6
j#:3
shipdate:70,1,31
quantity:300

s#:4
p#:6
j#:7
shipdate:70,1,31
quantity:300

s#:5
p#:2
j#:2
shipdate:70,1,23
quantity:200

s#:5
p#:2
j#:4
shipdate:70,1,23
quantity:100

s#:5
p#:3
j#:7
shipdate:70,1,31
quantity:100

s#:5
p#:5
j#:5
shipdate:70,1,12
quantity:500

s#:5
p#:5
j#:7
shipdate:70,1,12
quantity:100

s#:5
p#:6
j#:2
shipdate:70,1,16
quantity:200

s#:5
p#:1
j#:4
shipdate:70,2,29
quantity:100

s#:5
p#:3
j#:4
shipdate:70,1,14
quantity:200

s#:5
p#:4
j#:4
shipdate:70,1,24
quantity:800

s#:5
p#:5
j#:4
shipdate:70,1,12
quantity:400

s#:5
p#:6
j#:4
shipdate:70,2,5
quantity:500

B.2.2 websmpl.log in 2.2

223.98.45.100 - - [17/Oct/2002:15:19:11 +0900] "GET /cgi-bin/p03?id=A0027061 HTTP/1.0" 200 1305 4
109.151.249.228 - - [17/Oct/2002:15:19:11 +0900] "POST /cgi-bin/p02 HTTP/1.0" 200 1010 1
62.180.146.66 - - [17/Oct/2002:15:19:11 +0900] "GET /cgi-bin/p03?id=F1159172 HTTP/1.1" 200 3055 4
149.223.26.62 - - [17/Oct/2002:15:19:12 +0900] "GET /cgi-bin/p03?id=B4015628 HTTP/1.0" 200 1161 4
149.79.144.225 - - [17/Oct/2002:15:19:12 +0900] "GET /cgi-bin/p03?id=J0630286 HTTP/1.1" 200 1670 4
247.182.195.102 - - [17/Oct/2002:15:19:12 +0900] "GET / HTTP/1.1" 304 - 0
247.25.64.162 - - [17/Oct/2002:15:19:12 +0900] "GET /cgi-bin/p03?id=B4903411\" HTTP/1.1" 200 7217 12
62.77.228.32 - - [17/Oct/2002:15:19:13 +0900] "POST /cgi-bin/p02 HTTP/1.1" 200 979 2

B.2.3 Output from (2-15) in 2.3

num:223
num:98
num:45
num:100
num:17
num:2002
num:15
num:19
num:11
num:0900
num:03
num:0027061
num:1
num:0
num:200
num:1305
num:4

num:109
num:151
num:249
num:228
num:17
num:2002
num:15
num:19
num:11
num:0900
num:02
num:1
num:0
num:200
num:1010
num:1

num:62
num:180
num:146
num:66
num:17
num:2002
num:15
num:19
num:11
num:0900
num:03
num:1159172
num:1
num:1
num:200
num:3055
num:4

num:149
num:223
num:26
num:62
num:17
num:2002
num:15
num:19
num:12
num:0900
num:03
num:4015628
num:1
num:0
num:200
num:1161
num:4

num:149
num:79
num:144
num:225
num:17
num:2002
num:15
num:19
num:12
num:0900
num:03
num:0630286
num:1
num:1
num:200
num:1670
num:4

num:247
num:182
num:195
num:102
num:17
num:2002
num:15
num:19
num:12
num:0900
num:1
num:1
num:304
num:0

num:247
num:25
num:64
num:162
num:17
num:2002
num:15
num:19
num:12
num:0900
num:03
num:4903411
num:1
num:1
num:200
num:7217
num:12

num:62
num:77
num:228
num:32
num:17
num:2002
num:15
num:19
num:13
num:0900
num:02
num:1
num:1
num:200
num:979
num:2

B.2.4 mbox (2-19) in 2.4.2

From miyazawa@somewhere.jp  Fri Mar  1 20:10:48 2013
Return-Path: <miyazawa@somewhere.jp>
X-Original-To: miyazawa@elsewhere.jp
Delivered-To: miyazawa@elsewhere.jp
Received: from mailhost.somewhere.jp (mailhost.somewhere.jp [123.456.7.8])
by elsewhere.jp (Postfix) with ESMTP id 2B60D381428
for <miyazawa@elsewhere.jp>; Fri,  1 Mar 2013 20:10:48 +0900 (JST)
Received: from [123.456.45.43] (123.456.45.43 [123.456.45.43])
by mailhost.somewhere.jp (deepsmtpd.so 3.7.0)
with ESMTP id <61309CB0.50000@somewhere.jp>
for <miyazawa@elsewhere.jp>; Fri, 1 Mar 2013 20:10:40 +0900
Message-ID: <61309CB0.50000@somewhere.jp>
Date: Fri, 01 Mar 2013 20:10:40 +0900
From: MIYAZAWA Akira <miyazawa@somewhere.jp>
User-Agent: Mozilla/5.0 (X11; U; Linux i686; ja; rv:1.9.2.17) Gecko/20110414 SUSE/3.1.10 Thunderbird/3.1.10
MIME-Version: 1.0
To: miyazawa@elsewhere.jp
Subject: TEST mail
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-IP: 123.456.45.43
X-FROM-DOMAIN: somewhere.jp
X-FROM-EMAIL: miyazawa@somewhere.jp
Status: RO

HELLO WORLD!
THIS IS A SAMPLE MAIL BODY

From miyazawa@elsewhere.jp  Fri Mar  1 20:12:47 2013
Return-Path: <miyazawa@elsewhere.jp>
X-Original-To: miyazawa@elsewhere.jp
Delivered-To: miyazawa@elsewhere.jp
Received: by elsewhere.jp (Postfix, from userid 2508)
id 0A24638147A; Fri,  1 Mar 2013 20:12:47 +0900 (JST)
Date: Fri, 01 Mar 2013 20:12:47 +0900
To: miyazawa@somewhere.jp, miyazawa@elsewhere.jp
Subject: Re: TEST mail
References: <61309CB0.50000@somewhere.jp>
In-Reply-To: <61309CB0.50000@somewhere.jp>
User-Agent: Heirloom mailx 12.2 01/07/07
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-Id: <20130301111247.0A24638147A@elsewhere.jp>
From: miyazawa@elsewhere.jp (MIYAZAWA Akira)
Status: RO

HELLO! THIS IS REPLY.

From miyazawa@somewhere.jp  Fri Mar  1 20:14:59 2013
Return-Path: <miyazawa@somewhere.jp>
X-Original-To: miyazawa@elsewhere.jp
Delivered-To: miyazawa@elsewhere.jp
Received: from mailhost.somewhere.jp (mailhost.somewhere.jp [123.456.7.4])
by elsewhere.jp (Postfix) with ESMTP id 3BD3038159A
for <miyazawa@elsewhere.jp>; Fri,  1 Mar 2013 20:14:59 +0900 (JST)
Received: from [123.456.45.43] (123.456.45.43 [123.456.45.43])
by mailhost.somewhere.jp (deepsmtpd.so 3.7.0)
with ESMTP id <61309DA9.9060704@somewhere.jp>
for <miyazawa@elsewhere.jp>; Fri, 1 Mar 2013 20:14:49 +0900
Message-ID: <61309DA9.9060704@somewhere.jp>
Date: Fri, 01 Mar 2013 20:14:49 +0900
From: MIYAZAWA Akira <miyazawa@somewhere.jp>
User-Agent: Mozilla/5.0 (X11; U; Linux i686; ja; rv:1.9.2.17) Gecko/20110414 SUSE/3.1.10 Thunderbird/3.1.10
MIME-Version: 1.0
To: miyazawa@elsewhere.jp
Subject: Fwd: TEST mail
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-IP: 123.456.45.43
X-FROM-DOMAIN: somewhere.jp
X-FROM-EMAIL: miyazawa@somewhere.jp
Status: RO

THIS IS A REPLY.

-------- Original Message --------
Subject: TEST mail
Date: Fri, 01 Mar 2013 20:10:40 +0900
From: MIYAZAWA Akira <miyazawa@somewhere.jp>
To: miyazawa@elsewhere.jp

HELLO WORLD!
THIS IS A SAMPLE MAIL BODY

B.2.5 Full output from (2-19) in 2.4.2

From miyazawa@somewhere.jp  Fri Mar  1 20:10:48 2013
Return-Path:<miyazawa@somewhere.jp>
X-Original-To:miyazawa@elsewhere.jp
Delivered-To:miyazawa@elsewhere.jp
Received:from mailhost.somewhere.jp (mailhost.somewhere.jp [123.456.7.8]) by elsewhere.jp (Postfix) with ESMTP id 2B60D381428 for <miyazawa@elsewhere.jp>; Fri,  1 Mar 2013 20:10:48 +0900 (JST)
Received:from [123.456.45.43] (123.456.45.43 [123.456.45.43]) by mailhost.somewhere.jp (deepsmtpd.so 3.7.0) with ESMTP id <61309CB0.50000@somewhere.jp> for <miyazawa@elsewhere.jp>; Fri, 1 Mar 2013 20:10:40 +0900
Message-ID:<61309CB0.50000@somewhere.jp>
Date:Fri, 01 Mar 2013 20:10:40 +0900
From:MIYAZAWA Akira <miyazawa@somewhere.jp>
User-Agent:Mozilla/5.0 (X11; U; Linux i686; ja; rv:1.9.2.17) Gecko/20110414 SUSE/3.1.10 Thunderbird/3.1.10
MIME-Version:1.0
To:miyazawa@elsewhere.jp
Subject:TEST mail
Content-Type:text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding:7bit
X-IP:123.456.45.43
X-FROM-DOMAIN:somewhere.jp
X-FROM-EMAIL:miyazawa@somewhere.jp
Status:RO

From miyazawa@elsewhere.jp  Fri Mar  1 20:12:47 2013
Return-Path:<miyazawa@elsewhere.jp>
X-Original-To:miyazawa@elsewhere.jp
Delivered-To:miyazawa@elsewhere.jp
Received:by elsewhere.jp (Postfix, from userid 2508) id 0A24638147A; Fri,  1 Mar 2013 20:12:47 +0900 (JST)
Date:Fri, 01 Mar 2013 20:12:47 +0900
To:miyazawa@somewhere.jp, miyazawa@elsewhere.jp
Subject:Re: TEST mail
References:<61309CB0.50000@somewhere.jp>
In-Reply-To:<61309CB0.50000@somewhere.jp>
User-Agent:Heirloom mailx 12.2 01/07/07
MIME-Version:1.0
Content-Type:text/plain; charset=us-ascii
Content-Transfer-Encoding:7bit
Message-Id:<20130301111247.0A24638147A@elsewhere.jp>
From:miyazawa@elsewhere.jp (MIYAZAWA Akira)
Status:RO

From miyazawa@somewhere.jp  Fri Mar  1 20:14:59 2013
Return-Path:<miyazawa@somewhere.jp>
X-Original-To:miyazawa@elsewhere.jp
Delivered-To:miyazawa@elsewhere.jp
Received:from mailhost.somewhere.jp (mailhost.somewhere.jp [123.456.7.4]) by elsewhere.jp (Postfix) with ESMTP id 3BD3038159A for <miyazawa@elsewhere.jp>; Fri,  1 Mar 2013 20:14:59 +0900 (JST)
Received:from [123.456.45.43] (123.456.45.43 [123.456.45.43]) by mailhost.somewhere.jp (deepsmtpd.so 3.7.0) with ESMTP id <61309DA9.9060704@somewhere.jp> for <miyazawa@elsewhere.jp>; Fri, 1 Mar 2013 20:14:49 +0900
Message-ID:<61309DA9.9060704@somewhere.jp>
Date:Fri, 01 Mar 2013 20:14:49 +0900
From:MIYAZAWA Akira <miyazawa@somewhere.jp>
User-Agent:Mozilla/5.0 (X11; U; Linux i686; ja; rv:1.9.2.17) Gecko/20110414 SUSE/3.1.10 Thunderbird/3.1.10
MIME-Version:1.0
To:miyazawa@elsewhere.jp
Subject:Fwd: TEST mail
Content-Type:text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding:7bit
X-IP:123.456.45.43
X-FROM-DOMAIN:somewhere.jp
X-FROM-EMAIL:miyazawa@somewhere.jp
Status:RO

Subject:TEST mail
Date:Fri, 01 Mar 2013 20:10:40 +0900
From:MIYAZAWA Akira <miyazawa@somewhere.jp>
To:miyazawa@elsewhere.jp

B.3.1 Output from the third Dproj of (3-3) in 3.1

title:The Godfather
year:1972
star:Marlon Brando
star:Al Pacino

title:Singin' in the Rain
year:1952
star:Gene Kelly
star:Donald O'Connor
star:Debbie Reynolds

B.3.2 director.d (3-3) in 3.1

title:The Godfather
year:1972
director:Francis Ford Coppola

title:Singin' in the Rain
year:1952
director:Stanley Donen

title:Singin' in the Rain
year:1952
director:Gene Kelly

B.3.3 writer.d (3-3) in 3.1

title:The Godfather
year:1972
writer:Mario Puzo

title:The Godfather
year:1972
writer:Francis Ford Coppola

B.3.4 star.d (3-3) in 3.1

title:The Godfather
year:1972
star:Marlon Brando

title:The Godfather
year:1972
star:Al Pacino

title:Singin' in the Rain
year:1952
star:Gene Kelly

title:Singin' in the Rain
year:1952
star:Donald O'Connor

title:Singin' in the Rain
year:1952
star:Debbie Reynolds

B.3.5 tmp-director.d (3-5) in 3.2

title:The Godfather
year:1972
director:Francis Ford Coppola

title:Singin' in the Rain
year:1952
director:Stanley Donen
director:Gene Kelly

B.3.6 tmp-writer.d (3-5) in 3.2

title:The Godfather
year:1972
writer:Mario Puzo
writer:Francis Ford Coppola

B.3.7 tmp-star.d (3-5) in 3.2

title:The Godfather
year:1972
star:Marlon Brando
star:Al Pacino

title:Singin' in the Rain
year:1952
star:Gene Kelly
star:Donald O'Connor
star:Debbie Reynolds

B.3.8 star.d sorted by star in 3.3.1

title:The Godfather
year:1972
star:Al Pacino

title:Singin' in the Rain
year:1952
star:Debbie Reynolds

title:Singin' in the Rain
year:1952
star:Donald O'Connor

title:Singin' in the Rain
year:1952
star:Gene Kelly

title:The Godfather
year:1972
star:Marlon Brando

B.3.9 Result of (3-8) in 3.3.1

title:The Godfather
year:1972
star:Al Pacino

title:Singin' in the Rain
year:1952
star:Debbie Reynolds
star:Donald O'Connor
star:Gene Kelly

title:The Godfather
year:1972
star:Marlon Brando

B.3.10 Expected output from (3-9) in 3.3.2

title:The Godfather
year:1972
writer:Mario Puzo
writer:Francis Ford Coppola
director:Francis Ford Coppola
star:Marlon Brando
star:Al Pacino

title:Singin' in the Rain
year:1952
director:Stanley Donen
director:Gene Kelly
star:Gene Kelly
star:Donald O'Connor
star:Debbie Reynolds

B.3.11 Actual output from (3-9) in 3.3.2

title:The Godfather
year:1972
writer:Mario Puzo
writer:Francis Ford Coppola
director:Francis Ford Coppola
star:Marlon Brando
star:Al Pacino

B.3.12 new-director.d (3-11) in 3.3.2

rec#:1
fld#:1
title:The Godfather
year:1972
director:Francis Ford Coppola

rec#:2
fld#:1
title:Singin' in the Rain
year:1952
director:Stanley Donen

rec#:2
fld#:2
title:Singin' in the Rain
year:1952
director:Gene Kelly

B.3.13 new-writer.d (3-11) in 3.3.2

rec#:1
fld#:1
title:The Godfather
year:1972
writer:Mario Puzo

rec#:1
fld#:2
title:The Godfather
year:1972
writer:Francis Ford Coppola

B.3.14 new-star.d (3-11) in 3.3.2

rec#:1
fld#:1
title:The Godfather
year:1972
star:Marlon Brando

rec#:1
fld#:2
title:The Godfather
year:1972
star:Al Pacino

rec#:2
fld#:1
title:Singin' in the Rain
year:1952
star:Gene Kelly

rec#:2
fld#:2
title:Singin' in the Rain
year:1952
star:Donald O'Connor

rec#:2
fld#:3
title:Singin' in the Rain
year:1952
star:Debbie Reynolds

B.3.15 new-tmp-director.d (3-12) in 3.3.2

rec#:1
title:The Godfather
year:1972
director:Francis Ford Coppola

rec#:2
title:Singin' in the Rain
year:1952
director:Stanley Donen
director:Gene Kelly

B.3.16 new-tmp-writer.d (3-12) in 3.3.2

rec#:1
title:The Godfather
year:1972
writer:Mario Puzo
writer:Francis Ford Coppola

B.3.17 new-tmp-star.d (3-12) in 3.3.2

rec#:1
title:The Godfather
year:1972
star:Marlon Brando
star:Al Pacino

rec#:2
title:Singin' in the Rain
year:1952
star:Gene Kelly
star:Donald O'Connor
star:Debbie Reynolds

B.3.18 schumann-ops.d (3-20), (3-21a), (3-21b) in 3.5

op:14a
date:1836
title:Sonate No.3
atitle:Grosse Sonate
atitle:Concert ohne Orchester
version:Erste Ausgabe
category:keyboard
instrumentation:piano
pub:1836
part:1
tempo:Allegro brillante
key:f
part:2
ptitle:Quasi variazioni
tempo:Andantino de Clara Wieck
key:f
part:3
tempo:Prestissimo possibile
key:f

op:15
date:1838
title:Kinderscenen
category:keyboard
instrumentation:piano
pub:1839
part:1
ptitle:Von fremden Landern und Menschen
key:G
part:2
ptitle:Curiose Geschichte
key:D
part:3
ptitle:Hasche-Mann
key:h
part:4
ptitle:Bittendes Kind
key:D
part:5
ptitle:Glückes genug
key:D
part:6
ptitle:Wichtige Begebenheit
key:A
part:7
ptitle:Träumerei
key:F
part:8
ptitle:Am Camin
key:F
part:9
ptitle:Ritter vom Steckenpferd
key:C
part:10
ptitle:Fast zu ernst
key:gis
part:11
ptitle:Fürchtenmachen
key:G
part:12
ptitle:Kind im Einschlummern
key:e
part:13
ptitle:Der Dichter spricht
key:G

B.3.19 Output from (3-25) in 3.7

rec# j# p# pname color weight qoh s# sname loc    shipdate quantity
   1  4  1 Nut   Red       12  30  1 Smith London 70,1,12       700
                                   5 Adams Athens 70,2,9        100
   2  4  2 Bolt  Green     17  60  5 Adams Athens 70,1,23       100
   3  4  3 Screw Blue      17  30  2 Jones Paris  70,1,17       500
                                   5 Adams Athens 70,1,14       200
   4  4  4 Screw Red       14  10  5 Adams Athens 70,1,24       800
   5  4  5 Cam   Blue      12  20  5 Adams Athens 70,1,12       400
   6  4  6 Cog   Red       19  40  5 Adams Athens 70,2,5        500

B.3.20 Output from (3-26) in 3.7

                                      j#:1
                                  jname:Sorter
                                 mgr_e#:640028

rec# p# pname color weight qoh s# sname loc    shipdate quantity
   1  1 Nut   Red       12  30  1 Smith London 70,1,12       200
   2  3 Screw Blue      17  30  2 Jones Paris  70,1,17       400
                                3 Blake Paris  70,2,2        200


                                      j#:2
                                  jname:Punch
                                 mgr_e#:700128

rec# p# pname color weight qoh s# sname loc    shipdate quantity
   3  2 Bolt  Green     17  60  5 Adams Athens 70,1,23       200
   4  3 Screw Blue      17  30  2 Jones Paris  70,1,17       200
   5  4 Screw Red       14  10  3 Blake Paris  70,2,10       500
   6  5 Cam   Blue      12  20  2 Jones Paris  69,12,19      100
   7  6 Cog   Red       19  40  5 Adams Athens 70,1,16       200


                                      j#:3
                                  jname:Reader
                                 mgr_e#:680096

rec# p# pname color weight qoh s# sname loc    shipdate quantity
   8  3 Screw Blue      17  30  2 Jones Paris  70,1,17       200
   9  6 Cog   Red       19  40  4 Clark London 70,1,31       300


                                      j#:4
                                 jname:Console
                                 mgr_e#:660118

rec# p# pname color weight qoh s# sname loc    shipdate quantity
  10  1 Nut   Red       12  30  1 Smith London 70,1,12       700
                                5 Adams Athens 70,2,9        100
  11  2 Bolt  Green     17  60  5 Adams Athens 70,1,23       100
  12  3 Screw Blue      17  30  2 Jones Paris  70,1,17       500
                                5 Adams Athens 70,1,14       200
  13  4 Screw Red       14  10  5 Adams Athens 70,1,24       800
  14  5 Cam   Blue      12  20  5 Adams Athens 70,1,12       400
  15  6 Cog   Red       19  40  5 Adams Athens 70,2,5        500


                                      j#:5
                                 jname:Collator
                                 mgr_e#:690002

rec# p# pname color weight qoh s# sname loc    shipdate quantity
  16  3 Screw Blue      17  30  2 Jones Paris  70,1,17       600
  17  5 Cam   Blue      12  20  5 Adams Athens 70,1,12       500


                                      j#:6
                                 jname:Terminal
                                 mgr_e#:640014

rec# p# pname color weight qoh s# sname loc   shipdate quantity
  18  3 Screw Blue      17  30  2 Jones Paris 70,1,17       400


                                      j#:7
                                   jname:tape
                                 mgr_e#:710020

rec# p# pname color weight qoh s# sname loc    shipdate quantity
  19  3 Screw Blue      17  30  2 Jones Paris  70,1,17       800
                                5 Adams Athens 70,1,31       100
  20  5 Cam   Blue      12  20  5 Adams Athens 70,1,12       100
  21  6 Cog   Red       19  40  4 Clark London 70,1,31       300

B.3.21 spj.d (3-27) in 3.8

rec# s# p# j# shipdate quantity sname loc    pname color weight qoh jname    mgr_e#
   1  1  1  1 70,1,12       200 Smith London Nut   Red       12  30 Sorter   640028
   2  1  1  4 70,1,12       700 Smith London Nut   Red       12  30 Console  660118
   3  2  3  1 70,1,17       400 Jones Paris  Screw Blue      17  30 Sorter   640028
   4  2  3  2 70,1,17       200 Jones Paris  Screw Blue      17  30 Punch    700128
   5  2  3  3 70,1,17       200 Jones Paris  Screw Blue      17  30 Reader   680096
   6  2  3  4 70,1,17       500 Jones Paris  Screw Blue      17  30 Console  660118
   7  2  3  5 70,1,17       600 Jones Paris  Screw Blue      17  30 Collator 690002
   8  2  3  6 70,1,17       400 Jones Paris  Screw Blue      17  30 Terminal 640014
   9  2  3  7 70,1,17       800 Jones Paris  Screw Blue      17  30 tape     710020
  10  2  5  2 69,12,19      100 Jones Paris  Cam   Blue      12  20 Punch    700128
  11  3  3  1 70,2,2        200 Blake Paris  Screw Blue      17  30 Sorter   640028
  12  3  4  2 70,2,10       500 Blake Paris  Screw Red       14  10 Punch    700128
  13  4  6  3 70,1,31       300 Clark London Cog   Red       19  40 Reader   680096
  14  4  6  7 70,1,31       300 Clark London Cog   Red       19  40 tape     710020
  15  5  2  2 70,1,23       200 Adams Athens Bolt  Green     17  60 Punch    700128
  16  5  2  4 70,1,23       100 Adams Athens Bolt  Green     17  60 Console  660118
  17  5  3  7 70,1,31       100 Adams Athens Screw Blue      17  30 tape     710020
  18  5  5  5 70,1,12       500 Adams Athens Cam   Blue      12  20 Collator 690002
  19  5  5  7 70,1,12       100 Adams Athens Cam   Blue      12  20 tape     710020
  20  5  6  2 70,1,16       200 Adams Athens Cog   Red       19  40 Punch    700128
  21  5  1  4 70,2,9        100 Adams Athens Nut   Red       12  30 Console  660118
  22  5  3  4 70,1,14       200 Adams Athens Screw Blue      17  30 Console  660118
  23  5  4  4 70,1,24       800 Adams Athens Screw Red       14  10 Console  660118
  24  5  5  4 70,1,12       400 Adams Athens Cam   Blue      12  20 Console  660118
  25  5  6  4 70,2,5        500 Adams Athens Cog   Red       19  40 Console  660118

B.3.22 Output from (3-29) in 3.8

                                      s#:1
                                  sname:Smith
                                   loc:London

rec# p# pname color weight qoh j# jname   mgr_e# shipdate quantity
   1  1 Nut   Red       12  30  1 Sorter  640028 70,1,12       200
                                4 Console 660118 70,1,12       700


                                      s#:2
                                  sname:Jones
                                   loc:Paris

rec# p# pname color weight qoh j# jname    mgr_e# shipdate quantity
   2  3 Screw Blue      17  30  1 Sorter   640028 70,1,17       400
                                2 Punch    700128 70,1,17       200
                                3 Reader   680096 70,1,17       200
                                4 Console  660118 70,1,17       500
                                5 Collator 690002 70,1,17       600
                                6 Terminal 640014 70,1,17       400
                                7 tape     710020 70,1,17       800
   3  5 Cam   Blue      12  20  2 Punch    700128 69,12,19      100


                                      s#:3
                                  sname:Blake
                                   loc:Paris

rec# p# pname color weight qoh j# jname  mgr_e# shipdate quantity
   4  3 Screw Blue      17  30  1 Sorter 640028 70,2,2        200
   5  4 Screw Red       14  10  2 Punch  700128 70,2,10       500


                                      s#:4
                                  sname:Clark
                                   loc:London

rec# p# pname color weight qoh j# jname  mgr_e# shipdate quantity
   6  6 Cog   Red       19  40  3 Reader 680096 70,1,31       300
                                7 tape   710020 70,1,31       300


                                      s#:5
                                  sname:Adams
                                   loc:Athens

rec# p# pname color weight qoh j# jname    mgr_e# shipdate quantity
   7  1 Nut   Red       12  30  4 Console  660118 70,2,9        100
   8  2 Bolt  Green     17  60  2 Punch    700128 70,1,23       200
                                4 Console  660118 70,1,23       100
   9  3 Screw Blue      17  30  4 Console  660118 70,1,14       200
                                7 tape     710020 70,1,31       100
  10  4 Screw Red       14  10  4 Console  660118 70,1,24       800
  11  5 Cam   Blue      12  20  4 Console  660118 70,1,12       400
                                5 Collator 690002 70,1,12       500
                                7 tape     710020 70,1,12       100
  12  6 Cog   Red       19  40  2 Punch    700128 70,1,16       200
                                4 Console  660118 70,2,5        500

B.3.23 3-level.d: output from (3-30) in 3.8

s#:1
sname:Smith
loc:London
p#:1
pname:Nut
color:Red
weight:12
qoh:30
j#:1
jname:Sorter
mgr_e#:640028
shipdate:70,1,12
quantity:200
j#:4
jname:Console
mgr_e#:660118
shipdate:70,1,12
quantity:700

s#:2
sname:Jones
loc:Paris
p#:3
pname:Screw
color:Blue
weight:17
qoh:30
j#:1
jname:Sorter
mgr_e#:640028
shipdate:70,1,17
quantity:400
j#:2
jname:Punch
mgr_e#:700128
shipdate:70,1,17
quantity:200
j#:3
jname:Reader
mgr_e#:680096
shipdate:70,1,17
quantity:200
j#:4
jname:Console
mgr_e#:660118
shipdate:70,1,17
quantity:500
j#:5
jname:Collator
mgr_e#:690002
shipdate:70,1,17
quantity:600
j#:6
jname:Terminal
mgr_e#:640014
shipdate:70,1,17
quantity:400
j#:7
jname:tape
mgr_e#:710020
shipdate:70,1,17
quantity:800
p#:5
pname:Cam
color:Blue
weight:12
qoh:20
j#:2
jname:Punch
mgr_e#:700128
shipdate:69,12,19
quantity:100

s#:3
sname:Blake
loc:Paris
p#:3
pname:Screw
color:Blue
weight:17
qoh:30
j#:1
jname:Sorter
mgr_e#:640028
shipdate:70,2,2
quantity:200
p#:4
pname:Screw
color:Red
weight:14
qoh:10
j#:2
jname:Punch
mgr_e#:700128
shipdate:70,2,10
quantity:500

s#:4
sname:Clark
loc:London
p#:6
pname:Cog
color:Red
weight:19
qoh:40
j#:3
jname:Reader
mgr_e#:680096
shipdate:70,1,31
quantity:300
j#:7
jname:tape
mgr_e#:710020
shipdate:70,1,31
quantity:300

s#:5
sname:Adams
loc:Athens
p#:1
pname:Nut
color:Red
weight:12
qoh:30
j#:4
jname:Console
mgr_e#:660118
shipdate:70,2,9
quantity:100
p#:2
pname:Bolt
color:Green
weight:17
qoh:60
j#:2
jname:Punch
mgr_e#:700128
shipdate:70,1,23
quantity:200
j#:4
jname:Console
mgr_e#:660118
shipdate:70,1,23
quantity:100
p#:3
pname:Screw
color:Blue
weight:17
qoh:30
j#:4
jname:Console
mgr_e#:660118
shipdate:70,1,14
quantity:200
j#:7
jname:tape
mgr_e#:710020
shipdate:70,1,31
quantity:100
p#:4
pname:Screw
color:Red
weight:14
qoh:10
j#:4
jname:Console
mgr_e#:660118
shipdate:70,1,24
quantity:800
p#:5
pname:Cam
color:Blue
weight:12
qoh:20
j#:4
jname:Console
mgr_e#:660118
shipdate:70,1,12
quantity:400
j#:5
jname:Collator
mgr_e#:690002
shipdate:70,1,12
quantity:500
j#:7
jname:tape
mgr_e#:710020
shipdate:70,1,12
quantity:100
p#:6
pname:Cog
color:Red
weight:19
qoh:40
j#:2
jname:Punch
mgr_e#:700128
shipdate:70,1,16
quantity:200
j#:4
jname:Console
mgr_e#:660118
shipdate:70,2,5
quantity:500

B.3.24 Output from (3-36) in 3.9

<root>
<file>
 <group><jno>1</jno>
  <jname>Sorter</jname>
  <mgr_eno>640028</mgr_eno>
  <record>
   <pno>1</pno>
   <pname>Nut</pname>
   <color>Red</color>
   <weight>12</weight>
   <qoh>30</qoh>
   <leaf>
    <sno>1</sno>
    <sname>Smith</sname>
    <loc>London</loc>
    <shipdate>70,1,12</shipdate>
    <quantity>200</quantity>
   </leaf>
  </record>
  
  <record>
   <pno>3</pno>
   <pname>Screw</pname>
   <color>Blue</color>
   <weight>17</weight>
   <qoh>30</qoh>
   <leaf>
    <sno>2</sno>
    <sname>Jones</sname>
    <loc>Paris</loc>
    <shipdate>70,1,17</shipdate>
    <quantity>400</quantity>
   </leaf>
   <leaf>
    <sno>3</sno>
    <sname>Blake</sname>
    <loc>Paris</loc>
    <shipdate>70,2,2</shipdate>
    <quantity>200</quantity>
   </leaf>
  </record>
  
 </group>
 
 <group><jno>2</jno>
  <jname>Punch</jname>
  <mgr_eno>700128</mgr_eno>
  <record>
   <pno>2</pno>
   <pname>Bolt</pname>
   <color>Green</color>
   <weight>17</weight>
   <qoh>60</qoh>
   <leaf>
    <sno>5</sno>
    <sname>Adams</sname>
    <loc>Athens</loc>
    <shipdate>70,1,23</shipdate>
    <quantity>200</quantity>
   </leaf>
  </record>
 
  <record>
   <pno>3</pno>
   <pname>Screw</pname>
   <color>Blue</color>
   <weight>17</weight>
   <qoh>30</qoh>
   <leaf>
    <sno>2</sno>
    <sname>Jones</sname>
    <loc>Paris</loc>
    <shipdate>70,1,17</shipdate>
    <quantity>200</quantity>
   </leaf>
  </record>
 
  <record>
   <pno>4</pno>
   <pname>Screw</pname>
   <color>Red</color>
   <weight>14</weight>
   <qoh>10</qoh>
   <leaf>
    <sno>3</sno>
    <sname>Blake</sname>
    <loc>Paris</loc>
    <shipdate>70,2,10</shipdate>
    <quantity>500</quantity>
   </leaf>
  </record>
 
  <record>
   <pno>5</pno>
   <pname>Cam</pname>
   <color>Blue</color>
   <weight>12</weight>
   <qoh>20</qoh>
   <leaf>
    <sno>2</sno>
    <sname>Jones</sname>
    <loc>Paris</loc>
    <shipdate>69,12,19</shipdate>
    <quantity>100</quantity>
   </leaf>
  </record>
 
  <record>
   <pno>6</pno>
   <pname>Cog</pname>
   <color>Red</color>
   <weight>19</weight>
   <qoh>40</qoh>
   <leaf>
    <sno>5</sno>
    <sname>Adams</sname>
    <loc>Athens</loc>
    <shipdate>70,1,16</shipdate>
    <quantity>200</quantity>
   </leaf>
  </record>
 
 </group>
 
 <group><jno>3</jno>
  <jname>Reader</jname>
  <mgr_eno>680096</mgr_eno>
  <record>
   <pno>3</pno>
   <pname>Screw</pname>
   <color>Blue</color>
   <weight>17</weight>
   <qoh>30</qoh>
   <leaf>
    <sno>2</sno>
    <sname>Jones</sname>
    <loc>Paris</loc>
    <shipdate>70,1,17</shipdate>
    <quantity>200</quantity>
   </leaf>
  </record>
 
  <record>
   <pno>6</pno>
   <pname>Cog</pname>
   <color>Red</color>
   <weight>19</weight>
   <qoh>40</qoh>
   <leaf>
    <sno>4</sno>
    <sname>Clark</sname>
    <loc>London</loc>
    <shipdate>70,1,31</shipdate>
    <quantity>300</quantity>
   </leaf>
  </record>
 
 </group>
 
 <group><jno>4</jno>
  <jname>Console</jname>
  <mgr_eno>660118</mgr_eno>
  <record>
   <pno>1</pno>
   <pname>Nut</pname>
   <color>Red</color>
   <weight>12</weight>
   <qoh>30</qoh>
   <leaf>
    <sno>1</sno>
    <sname>Smith</sname>
    <loc>London</loc>
    <shipdate>70,1,12</shipdate>
    <quantity>700</quantity>
   </leaf>
   <leaf>
    <sno>5</sno>
    <sname>Adams</sname>
    <loc>Athens</loc>
    <shipdate>70,2,9</shipdate>
    <quantity>100</quantity>
   </leaf>
  </record>
 
  <record>
   <pno>2</pno>
   <pname>Bolt</pname>
   <color>Green</color>
   <weight>17</weight>
   <qoh>60</qoh>
   <leaf>
    <sno>5</sno>
    <sname>Adams</sname>
    <loc>Athens</loc>
    <shipdate>70,1,23</shipdate>
    <quantity>100</quantity>
   </leaf>
  </record>
 
  <record>
   <pno>3</pno>
   <pname>Screw</pname>
   <color>Blue</color>
   <weight>17</weight>
   <qoh>30</qoh>
   <leaf>
    <sno>2</sno>
    <sname>Jones</sname>
    <loc>Paris</loc>
    <shipdate>70,1,17</shipdate>
    <quantity>500</quantity>
   </leaf>
   <leaf>
    <sno>5</sno>
    <sname>Adams</sname>
    <loc>Athens</loc>
    <shipdate>70,1,14</shipdate>
    <quantity>200</quantity>
   </leaf>
  </record>
 
  <record>
   <pno>4</pno>
   <pname>Screw</pname>
   <color>Red</color>
   <weight>14</weight>
   <qoh>10</qoh>
   <leaf>
    <sno>5</sno>
    <sname>Adams</sname>
    <loc>Athens</loc>
    <shipdate>70,1,24</shipdate>
    <quantity>800</quantity>
   </leaf>
  </record>
 
  <record>
   <pno>5</pno>
   <pname>Cam</pname>
   <color>Blue</color>
   <weight>12</weight>
   <qoh>20</qoh>
   <leaf>
    <sno>5</sno>
    <sname>Adams</sname>
    <loc>Athens</loc>
    <shipdate>70,1,12</shipdate>
    <quantity>400</quantity>
   </leaf>
  </record>
 
  <record>
   <pno>6</pno>
   <pname>Cog</pname>
   <color>Red</color>
   <weight>19</weight>
   <qoh>40</qoh>
   <leaf>
    <sno>5</sno>
    <sname>Adams</sname>
    <loc>Athens</loc>
    <shipdate>70,2,5</shipdate>
    <quantity>500</quantity>
   </leaf>
  </record>
 
 </group>
 
 <group><jno>5</jno>
  <jname>Collator</jname>
  <mgr_eno>690002</mgr_eno>
  <record>
   <pno>3</pno>
   <pname>Screw</pname>
   <color>Blue</color>
   <weight>17</weight>
   <qoh>30</qoh>
   <leaf>
    <sno>2</sno>
    <sname>Jones</sname>
    <loc>Paris</loc>
    <shipdate>70,1,17</shipdate>
    <quantity>600</quantity>
   </leaf>
  </record>
 
  <record>
   <pno>5</pno>
   <pname>Cam</pname>
   <color>Blue</color>
   <weight>12</weight>
   <qoh>20</qoh>
   <leaf>
    <sno>5</sno>
    <sname>Adams</sname>
    <loc>Athens</loc>
    <shipdate>70,1,12</shipdate>
    <quantity>500</quantity>
   </leaf>
  </record>
 
 </group>
 
 <group><jno>6</jno>
  <jname>Terminal</jname>
  <mgr_eno>640014</mgr_eno>
  <record>
   <pno>3</pno>
   <pname>Screw</pname>
   <color>Blue</color>
   <weight>17</weight>
   <qoh>30</qoh>
   <leaf>
    <sno>2</sno>
    <sname>Jones</sname>
    <loc>Paris</loc>
    <shipdate>70,1,17</shipdate>
    <quantity>400</quantity>
   </leaf>
  </record>
 
 </group>
 
 <group><jno>7</jno>
  <jname>tape</jname>
  <mgr_eno>710020</mgr_eno>
  <record>
   <pno>3</pno>
   <pname>Screw</pname>
   <color>Blue</color>
   <weight>17</weight>
   <qoh>30</qoh>
   <leaf>
    <sno>2</sno>
    <sname>Jones</sname>
    <loc>Paris</loc>
    <shipdate>70,1,17</shipdate>
    <quantity>800</quantity>
   </leaf>
   <leaf>
    <sno>5</sno>
    <sname>Adams</sname>
    <loc>Athens</loc>
    <shipdate>70,1,31</shipdate>
    <quantity>100</quantity>
   </leaf>
  </record>
 
  <record>
   <pno>5</pno>
   <pname>Cam</pname>
   <color>Blue</color>
   <weight>12</weight>
   <qoh>20</qoh>
   <leaf>
    <sno>5</sno>
    <sname>Adams</sname>
    <loc>Athens</loc>
    <shipdate>70,1,12</shipdate>
    <quantity>100</quantity>
   </leaf>
  </record>
 
  <record>
   <pno>6</pno>
   <pname>Cog</pname>
   <color>Red</color>
   <weight>19</weight>
   <qoh>40</qoh>
   <leaf>
    <sno>4</sno>
    <sname>Clark</sname>
    <loc>London</loc>
    <shipdate>70,1,31</shipdate>
    <quantity>300</quantity>
   </leaf>
  </record>
 
 </group>
</file>
 
</root>

B.3.25 Output from (3-37) in 3.9

<root>
<file>
 <group1><sno>1</sno>
  <sname>Smith</sname>
  <loc>London</loc>
  <group2><pno>1</pno>
   <pname>Nut</pname>
   <color>Red</color>
   <weight>12</weight>
   <qoh>30</qoh>
   <record>
    <jno>1</jno>
    <jname>Sorter</jname>
    <mgr_eno>640028</mgr_eno>
    <shipdate>70,1,12</shipdate>
    <quantity>200</quantity>
   </record>

   <record>
    <jno>4</jno>
    <jname>Console</jname>
    <mgr_eno>660118</mgr_eno>
    <shipdate>70,1,12</shipdate>
    <quantity>700</quantity>
   </record>

  </group2>

 </group1>

 <group1><sno>2</sno>
  <sname>Jones</sname>
  <loc>Paris</loc>
  <group2><pno>3</pno>
   <pname>Screw</pname>
   <color>Blue</color>
   <weight>17</weight>
   <qoh>30</qoh>
   <record>
    <jno>1</jno>
    <jname>Sorter</jname>
    <mgr_eno>640028</mgr_eno>
    <shipdate>70,1,17</shipdate>
    <quantity>400</quantity>
   </record>

   <record>
    <jno>2</jno>
    <jname>Punch</jname>
    <mgr_eno>700128</mgr_eno>
    <shipdate>70,1,17</shipdate>
    <quantity>200</quantity>
   </record>

   <record>
    <jno>3</jno>
    <jname>Reader</jname>
    <mgr_eno>680096</mgr_eno>
    <shipdate>70,1,17</shipdate>
    <quantity>200</quantity>
   </record>

   <record>
    <jno>4</jno>
    <jname>Console</jname>
    <mgr_eno>660118</mgr_eno>
    <shipdate>70,1,17</shipdate>
    <quantity>500</quantity>
   </record>

   <record>
    <jno>5</jno>
    <jname>Collator</jname>
    <mgr_eno>690002</mgr_eno>
    <shipdate>70,1,17</shipdate>
    <quantity>600</quantity>
   </record>

   <record>
    <jno>6</jno>
    <jname>Terminal</jname>
    <mgr_eno>640014</mgr_eno>
    <shipdate>70,1,17</shipdate>
    <quantity>400</quantity>
   </record>

   <record>
    <jno>7</jno>
    <jname>tape</jname>
    <mgr_eno>710020</mgr_eno>
    <shipdate>70,1,17</shipdate>
    <quantity>800</quantity>
   </record>

  </group2>

  <group2><pno>5</pno>
   <pname>Cam</pname>
   <color>Blue</color>
   <weight>12</weight>
   <qoh>20</qoh>
   <record>
    <jno>2</jno>
    <jname>Punch</jname>
    <mgr_eno>700128</mgr_eno>
    <shipdate>69,12,19</shipdate>
    <quantity>100</quantity>
   </record>

  </group2>

 </group1>

 <group1><sno>3</sno>
  <sname>Blake</sname>
  <loc>Paris</loc>
  <group2><pno>3</pno>
   <pname>Screw</pname>
   <color>Blue</color>
   <weight>17</weight>
   <qoh>30</qoh>
   <record>
    <jno>1</jno>
    <jname>Sorter</jname>
    <mgr_eno>640028</mgr_eno>
    <shipdate>70,2,2</shipdate>
    <quantity>200</quantity>
   </record>

  </group2>

  <group2><pno>4</pno>
   <pname>Screw</pname>
   <color>Red</color>
   <weight>14</weight>
   <qoh>10</qoh>
   <record>
    <jno>2</jno>
    <jname>Punch</jname>
    <mgr_eno>700128</mgr_eno>
    <shipdate>70,2,10</shipdate>
    <quantity>500</quantity>
   </record>

  </group2>

 </group1>

 <group1><sno>4</sno>
  <sname>Clark</sname>
  <loc>London</loc>
  <group2><pno>6</pno>
   <pname>Cog</pname>
   <color>Red</color>
   <weight>19</weight>
   <qoh>40</qoh>
   <record>
    <jno>3</jno>
    <jname>Reader</jname>
    <mgr_eno>680096</mgr_eno>
    <shipdate>70,1,31</shipdate>
    <quantity>300</quantity>
   </record>

   <record>
    <jno>7</jno>
    <jname>tape</jname>
    <mgr_eno>710020</mgr_eno>
    <shipdate>70,1,31</shipdate>
    <quantity>300</quantity>
   </record>

  </group2>

 </group1>

 <group1><sno>5</sno>
  <sname>Adams</sname>
  <loc>Athens</loc>
  <group2><pno>1</pno>
   <pname>Nut</pname>
   <color>Red</color>
   <weight>12</weight>
   <qoh>30</qoh>
   <record>
    <jno>4</jno>
    <jname>Console</jname>
    <mgr_eno>660118</mgr_eno>
    <shipdate>70,2,9</shipdate>
    <quantity>100</quantity>
   </record>

  </group2>

  <group2><pno>2</pno>
   <pname>Bolt</pname>
   <color>Green</color>
   <weight>17</weight>
   <qoh>60</qoh>
   <record>
    <jno>2</jno>
    <jname>Punch</jname>
    <mgr_eno>700128</mgr_eno>
    <shipdate>70,1,23</shipdate>
    <quantity>200</quantity>
   </record>

   <record>
    <jno>4</jno>
    <jname>Console</jname>
    <mgr_eno>660118</mgr_eno>
    <shipdate>70,1,23</shipdate>
    <quantity>100</quantity>
   </record>

  </group2>

  <group2><pno>3</pno>
   <pname>Screw</pname>
   <color>Blue</color>
   <weight>17</weight>
   <qoh>30</qoh>
   <record>
    <jno>4</jno>
    <jname>Console</jname>
    <mgr_eno>660118</mgr_eno>
    <shipdate>70,1,14</shipdate>
    <quantity>200</quantity>
   </record>

   <record>
    <jno>7</jno>
    <jname>tape</jname>
    <mgr_eno>710020</mgr_eno>
    <shipdate>70,1,31</shipdate>
    <quantity>100</quantity>
   </record>

  </group2>

  <group2><pno>4</pno>
   <pname>Screw</pname>
   <color>Red</color>
   <weight>14</weight>
   <qoh>10</qoh>
   <record>
    <jno>4</jno>
    <jname>Console</jname>
    <mgr_eno>660118</mgr_eno>
    <shipdate>70,1,24</shipdate>
    <quantity>800</quantity>
   </record>

  </group2>

  <group2><pno>5</pno>
   <pname>Cam</pname>
   <color>Blue</color>
   <weight>12</weight>
   <qoh>20</qoh>
   <record>
    <jno>4</jno>
    <jname>Console</jname>
    <mgr_eno>660118</mgr_eno>
    <shipdate>70,1,12</shipdate>
    <quantity>400</quantity>
   </record>

   <record>
    <jno>5</jno>
    <jname>Collator</jname>
    <mgr_eno>690002</mgr_eno>
    <shipdate>70,1,12</shipdate>
    <quantity>500</quantity>
   </record>

   <record>
    <jno>7</jno>
    <jname>tape</jname>
    <mgr_eno>710020</mgr_eno>
    <shipdate>70,1,12</shipdate>
    <quantity>100</quantity>
   </record>

  </group2>

  <group2><pno>6</pno>
   <pname>Cog</pname>
   <color>Red</color>
   <weight>19</weight>
   <qoh>40</qoh>
   <record>
    <jno>2</jno>
    <jname>Punch</jname>
    <mgr_eno>700128</mgr_eno>
    <shipdate>70,1,16</shipdate>
    <quantity>200</quantity>
   </record>

   <record>
    <jno>4</jno>
    <jname>Console</jname>
    <mgr_eno>660118</mgr_eno>
    <shipdate>70,2,5</shipdate>
    <quantity>500</quantity>
   </record>

  </group2>
 </group1>
</file>

</root>