Matrix import wizard for sample data

After selecting Target: Sampling event a window as shown below will be displayed. The window is separated in 3 areas. On the left side you see the descriptor settings for the actual file column (presently not visible). On the right side you see the list of available import steps. In the middle part the details of the selected import steps are shown.

 

Choosing the File

As a first step, choose the File from where the data should be imported. The currently supported format is tab-separated text. Then choose the Encoding of the file, e.g. Unicode.  The Header lines specifies the number of header lines, usually at least the first line which typically contains the descriptor names (see image below). The Start line and End line will automatically be set according to your data. You may change these to restrict the data lines, i.e. the descriptions/items that shall be imported. The not imported parts in the file are indicated with a gray background. If your data contains e.g. date information or floating point values where notations differ between countries (e.g. 3.14 - 3,13), choose the Language / Country to ensure a correct interpretation of your data. Finally you can select a prepared Schema (see sections Testing and Import below) for the import.

 

Choosing the descriptor types

Choosing of the descriptor types is done in exactly the same way as described in section Matrix wizard for description data.

 

Table data

To set the source for the data columns, select the step of a table listed underneath the File step. Be aware that for the Descriptor table not the file columns will be selected but the file rows in the header lines. In the image below you see an example of the descriptor data columns, where the values for id and label are taken from the first line in the file. Additionally the descriptors will be assigned to the descriptor tree "Descriptor tree for matrix".  

Remark: Although descriptortree_id is a numeric value in the database, the externally visible descriptor tree name is expected here. Since this name is unambigious, the corresponding id will be determined during import.

The option Address index allows the selection of the column number (for descriptor) rsp. the line number (for description or sampling event) as value. See below an example for the Description table data columns. For id and label the For all:  alue "++ New description ++" was selected to create a single description entry where all sample data are attached. Finally the description is assigned to the project "Matrix text sample".

Remark: Although project_id is a numeric value in the database, the externally visible project name is expected here. Since this name is unambigious, the corresponding id will be determined during import.

Finally the source for the Sampling event table data columns must be selected. For id and label the file column 0 has been selected (see image below).

A reminder in the header line will show you what actions are still needed to import the data into the table:

The handling of the columns is almost the same as described in the chapter columns of the table oriented import wizard.

 

Testing

To test if all requirements for the import are met use the Testing step. You can navigate to a certain data cell by using the Column: and Line: controls. As an alternative select the data cell ent click on button . Finally click on the Test data button. If there are still unmet requirements, these will be listed in a window as shown below. In the window below you can see the following test actions:

  1. Insert of a quantitative descriptor "Channel 2 VLT Us1G"
  2. Insert of a descriptor tree node (assignemnt to tree "Tree for matrix test sample")
  3. Insert of a recommended statistical measure (measure "Mean") for the descriptor tree node
  4. Insert of description "++ New description ++" for project "Matrix test sample"
  5. Insert of a sampling event "30.01.2014 22:00" for description "++ New description ++"
  6. Insert of a sampling unit with id 199 for sampling event "30.01.2014 22:00"
  7. Insert of a quantitative value -22.72 to sampling unit 199

You may save the schema file by a click on button Save schema. If you click on button Generate flat files according to your adjustments data and matching schema files for the table oriented import wizards will be generated. This option may be used, if you want to import additional data, e.g. a descriptor's measurement unit, that are not supported by the matrix wizard. Since all mapping relevant data are stored in the selected import session, you may import the basic descriptor and description data using the matrix wizard and append additional data with the table oriented import wizards. In this case the generated flat data and schema files might be useful to have a defined starting point.    

 

Import

With the last step you can finally start to import the data into the database. If you want to repeat the import with the same settings and data of the same structure, you can save a schema of the current settings (see below).

Schedule for matrix import of tab-separated text files into DiversityDescriptions
Target within DiversityDescriptions: SamplingEvent
Schedule version: 1 Database version: 03.00.17
Lines: 5 - 151 Header lines: 4
Encoding: ANSI Language: US

Tables

Descriptor (Descriptor)
Merge handling: Merge
Column in table ? Copy Pre Post File pos. Value Source
id ? 1 File
label Channel 1 File
+ 2 File
+ 3 File
descriptortree_id Tree for matrix test sample Interface

Description (Description)
Merge handling: Merge
Column in table ? Copy Pre Post File pos. Value Source
id ? ++ New description ++ Interface
label ++ New description ++ Interface
project_id Matrix test sample Interface

SamplingEvent (SamplingEvent)
Merge handling: Merge
Column in table ? Copy Pre Post File pos. Value Source
id ? 0 File
label 0 File

Descriptor columns

File pos. Descriptor type Additional data
2 Quantitative
Statistical measure
Mean (= average)
3 Quantitative
Statistical measure
Mean (= average)
4 Quantitative
Statistical measure
Mean (= average)
5 Quantitative
Statistical measure
Mean (= average)
6 Quantitative
Statistical measure
Mean (= average)
7 Quantitative
Statistical measure
Mean (= average)
8 Quantitative
Statistical measure
Mean (= average)
9 Quantitative
Statistical measure
Mean (= average)
10 Quantitative
Statistical measure
Mean (= average)
11 Quantitative
Statistical measure
Mean (= average)
12 Quantitative
Statistical measure
Mean (= average)
13 Quantitative
Statistical measure
Mean (= average)
14 Quantitative
Statistical measure
Mean (= average)
15 Quantitative
Statistical measure
Mean (= average)
16 Quantitative
Statistical measure
Mean (= average)
17 Quantitative
Statistical measure
Mean (= average)
18 Quantitative
Statistical measure
Mean (= average)
19 Quantitative
Statistical measure
Mean (= average)
20 Quantitative
Statistical measure
Mean (= average)
21 Quantitative
Statistical measure
Mean (= average)
22 Quantitative
Statistical measure
Mean (= average)
23 Quantitative
Statistical measure
Mean (= average)
24 Quantitative
Statistical measure
Mean (= average)
25 Quantitative
Statistical measure
Mean (= average)
26 Quantitative
Statistical measure
Mean (= average)
27 Quantitative
Statistical measure
Mean (= average)
28 Quantitative
Statistical measure
Mean (= average)
29 Quantitative
Statistical measure
Mean (= average)
30 Quantitative
Statistical measure
Mean (= average)
31 Quantitative
Statistical measure
Mean (= average)
32 Quantitative
Statistical measure
Mean (= average)
33 Quantitative
Statistical measure
Mean (= average)
34 Quantitative
Statistical measure
Mean (= average)
35 Quantitative
Statistical measure
Mean (= average)
36 Quantitative
Statistical measure
Mean (= average)
37 Quantitative
Statistical measure
Mean (= average)
38 Quantitative
Statistical measure
Mean (= average)
39 Quantitative
Statistical measure
Mean (= average)
40 Quantitative
Statistical measure
Mean (= average)
41 Quantitative
Statistical measure
Mean (= average)
42 Quantitative
Statistical measure
Mean (= average)
43 Quantitative
Statistical measure
Mean (= average)
44 Quantitative
Statistical measure
Mean (= average)
45 Quantitative
Statistical measure
Mean (= average)

File cells that could not be imported will be marked with a red background while imported lines are marked green. If you want to save lines that produce errors during the import in a separate file, use the Save failed lines option. The protocol of the import will contain all settings acording to the used schema and an overview containing the number of inserted, updated, unchanged and failed lines (see below).

Protocol

Responsible: Link (DB-User: Workshop)
Date: Donnerstag, 20. April 2017, 16:17:42
Server: training.diversityworkbench.de
Database: DiversityDescriptions_Workshop
Descriptor columns total: 44
Descriptors imported: 44
Import lines total: 147
Descriptions imported: 1
Samples imported: 147
Cells imported: 6348
Cells failed: 120