Chapter 4 of the VDPMill User Guide
VDPMill Document Processing
3.1. Processing Basics
The logical flow of document processing can be divided into three phases:
-
Pre-formatting - includes pre-split XSLT, splitting, post-split XSLT.
-
Formatting - includes embedded XSLT and XSL-FO to XEPOUT rendering.
-
Post-formatting - includes custom post-processing, separated and joined output files generation.
Pre-formatting steps are the following:
-
Pre-split XSLT - An XSLT transformation used to customize the source file for special business rule or functional requirements during the formatting run. For example, the stylesheet may insert processing instructions in the source for splitting the document.
-
Splitting - The file is split according to its structure and splitting rules. A document is split either by element or by processing instructions in the source.
-
Post-split XSLT - An XSLT transformation used to generate an XSL-FO file from the individual split XML source files. This is convenient when the source xml is very large and contains batches of data. For example, an xml source with many customer invoices.
-
Formatting - XSL-FO is fed into the formatter, which creates and fills pages according to the specification defined in the XSL-FO document. The results of the formatting stage are XEPOUT files to be processed later. If
formatter-typein VDPMill configuration isenmasse, VDPMill uses EnMasse Toaster to format the documents. In this case an embedded XSLT can be applied to the XML file if it contains a reference to stylesheet via <xml-stylesheet ...> processing instruction -
Custom Post-processing - User-defined post-processing of XEPOUT intermediate document before the generation and joining phases. See Appendix D, Document Post-processing**
-
Joining - Concatenation of XEPOUT documents together. If splitting has been applied, generated chunk XEPOUT documents are joined and one joined output file is generated for one input file. If no splitting, one output file is generated for all the source files.
-
Separated Generation - XEPOUT file is converted to the requested output format supported by RenderX IREn. For more detailed information, see IREn User Guide.
XSLT transforms a user-defined XML document to another XML document. Several tasks can be done in this stage of processing (adding barcodes to a document, etc). Basically XSLT is used to generate XSL-FO files from XML documents.
Splitting divides a file into smaller chunks. The content and size of the chunks depend on the source file and the splitting rules. The split chunks can be formatted and then delivered in a printable format. They can be joined after the formatting into one large file. Splitting large files helps to avoid out of memory errors on large file formatting processes.
Formatting converts XSL-FO files to the XEPOUT format. If the source file contains a reference to a stylesheet, or an external stylesheet specified (in the GUI, from the command line, or via VDPMill API), XSLT is done before formatting.
Custom Post-processing can be used to customize a formatted document before generation (adding watermarks to a document, applying transpromo, filtering OMR marks or any user-defined actions).
The Joining functionality can be used to get one large output file after Splitting and Formatting and generate large printable documents. It also can be used to concatenate XEPOUT files from other sources.
Separated output generation makes printable documents /PDF, PostScript, AFP/ from XEPOUT files.
3.2. Splitting
3.2.1. When Splitting is Useful
Large files can cause out of memory errors during formatting. Splitting allows dividing a large file into small ones and formatting them simultaneously, which makes formatting process faster.
An XML file may contain collected information about similar but different things. It can be split to be delivered as a collection of separate printable files.
3.2.2. Splitting Basics
An XML file contains two parts:
-
Common parts - content that should appear in each split chunk,
-
Parts to be split - content that should be split into chunks.
For example, here is an XSL-FO file, where the part to be split (split area) contains the fo:block elements with their content and the common parts in this file are:
-
Header - the content from the beginning of the file to the
<fo:flow>start tag included; -
Footer - the content from the
</fo:flow>end tag to the end of the file;
|
Let's introduce two splitting terms:
-
Splitting start point - a place in an XML file which separates the head common part from the split area.
-
Split points - places where data in split area can be separated.
In the example above, the split start point lies after the <fo:flow> start tag. Split points can be placed between <fo:block> elements. For example, if four chunks are needed, it means that each chunk will contain one fo:block element, and the split files will be the following:
|
...
|
Important: Important: Currently, only end tags can be repeated in split files.
Important: Important: Split start point and split points must occur on the same level in an XML file with the single pass splitter (Splitter 1g).
3.2.3. How to Set Splitting Start Point and Split Points in an XML File
VDPMill provides two ways to define a split start point, as well as two ways to define split point. For details see Section 2.3, “splitter.conf Description”.
3.2.3.1. Defining a Split Start Point
A split start point can be defined in the XML files as:
-
PI
-
Start tag
3.2.3.2. Defining Split Point
A split point can be defined in an XML files as:
-
PI
-
Number of elements (element count) that need to be included in each chunk
3.2.4. Splitter 2g - Double Pass Splitter
Splitter 2g can be used to split an XML file which contains split points on mixed levels in the document. It works similarly to the single pass splitter - Splitter 1g (see Section 3.2.2, “Splitting Basics”) - except that the document can be split into chunks constructed from different level data. Here is an example:
|
The chunks generated from the example above will be the following:
|
|
|
|
3.2.5. Final Format Generation Control using XML PIs
If an XML files has been split it may be important to control the appearance of the chunk in the final output. For these purposes VDPMill uses XML PIs to get information of final formats for separated and joined output files. There are two XML PIs to control joined and separated output generation from the chunk documents: <?xepx-vdpmill-skip-join format?> <?xepx-vdpmill-skip-gen format?> where format is one of output formats available in IREn. <?xepx-vdpmill-skip-join format?> means that the chunk document which contains the PI should skip joining for format. <?xepx-vdpmill-skip-gen format?> means to skip the generation of a separated document in format.
3.2.6. Problems Caused by Splitting
Cross-references can cause problems, if the referenced destination does not lie in the same chunk as the reference itself.
3.2.7. Examples
Consider the XSL-FO example file from Section 3.2.2, “Splitting Basics”. The file should be split into chunks containing three <fo:block> elements each. In this case, splitting start point can be defined in that file as a processing instruction and a split point should be defined after the third <fo:block>.
|
The appropriate splitter configuration file should be the following:
|
| (1) | Defined split start point as a processing instruction. |
| (2) | Defined split point as three elements in each split chunk. |
3.3. Formatting and XSLT
VDPMill uses RenderX IREn and EnMasse to format XSL-FO files.
The XML file will be transformed before formatting if it contains a reference to a stylesheet. This option is supported only if the formatter-type is enmasse (See Section 2.2, “ vdpmill.conf Description”). VDPMill also allows pre- and post- split xsl transformations (See Pre-split XSLT and Post-split XSLT points in Chapter 3, VDPMill Document Processing).
Important: Important: The EnMasse instance used by VDPMill Formatter should be configured to run as Toaster and to produce XEPOUT files. For details see the EnMasse documentation. Running the EnMasse instance used by VDPMill on the same physical machine is the best option, because this ensures correct processing of document external resources (stylesheets, images) which appear in the documents as relative URIs.
3.4. Custom Post-processing
XEPOUT documents can be processed specially before final format files are generated. Since XEPOUT documents are in XML, a special post-processor can handle them and pass modified XEPOUT document to the final format generator or joiner.
For the detailed description of document post-processing see Appendix D, Document Post-processing**.
3.5. Joining XEPOUT Files
Some printers can accept large files (PostScript, AFP). Large print files can be generated with VDPMill by joining XEPOUT chunks. Simply, XEPOUT files are concatenated by adding the pages sequence from the second file to the first and so on. In the example below there are three separated XEPOUT files and the joining of these files is the result:
the separated files:
|
|
|
the result:
|
VDPMill supports flexible Join operations. For example, it is possible to get joined PDF output, separated PS, joined AFP, joined and separated XEPOUT files (see Chapter 4, VDPMill Interactive GUI Application and Chapter 9, VDPMill Command Line Interface).
Chunk document appearance in the joined output files can be controlled using special XML PIs. See Section 3.2.5, “Final Format Generation Control using XML PIs”.
3.5.1. Problems Caused by Joining
The main problem of joining is page numbering. VDPMill Joiner doesn't adjust page numbers in joined chunks to allow cross-references to work.
3.6. Separated Output File Generation
The Generation process makes printable files from chunk XEPOUT documents. For available output formats and final format options see IREn User Guide.
Chunk document appearance in the joined output files can be controlled using special XML PIs. See Section 3.2.5, “Final Format Generation Control using XML PIs”.