Export data as SDD, DELTA or EML

With this form you can export data from the database to an DELTA file. Choose Data -> Export -> Export SDD/DELTA ... from the menu to open the window for the export.

 

In the Export project section all projects of the database are shown as a tree. Select here the project that shall be exportes. In case of hierarchically organized projects the subordinated projects will be included for export, if the Include child projects option is checked. You may pass a description list to the form by starting a query in mode "Edit descriptions". Now you have the option to select single descriptions for export. In this case the Export project section shows the button  to switch to the list view (see below).

 

In the Export descriptions section you find all description titles that have been passed to the export form (see below). You may select all entries by clicking the all button, deselect all entries by clicking the none button or toggle your selection by clicking the swap button. By clicking the button you will return to the Export project view.

The Withheld data section allows control over export of datasets that contain entries with data status "Data withheld". Option Supress whole description (default) excludes all descriptions form export where at least on descriptor is marked with "Data withheld". Option Hide withheld descriptor excludes only the corresponding descriptor data from the description. Option Export withheld data does not exclude any data from export. Check option Export sample data to include sampling events and units in the analysis output.

In the Options section you find the items Read external data. By selecting this item you enable access to external databases linked by description scopes or the related project. The foreign data are included in the analysis tree and may be exported, e.g. to a SDD file. You may check the external data connections by clicking the button.

If Check strings for illegal characters in the Options section is selected, all string literals that shall be exported from database are scanned for illegal non-printable characters and matches are replaced by a double exclamation mark ("‼") in the output file. Although DiversityDescriptions can handle non-printable characters, especially in case of SDD XML files some applications may have problems opening files with illegal characters. 

With option Include data entry notes you allow the analysis step to store descriptor's data entry notes in the analysis tree. Depending on the export format, these data may be exported e.g. as "internal notes" (SDD).  

With option Include applicability data you allow the analysis step to write available descriptor applicability data to the analysis tree as a dedicated "Dependency tree".  

With option Include resources you allow the analysis step to write availble resource data to the analysis tree. This option is not relevant if you only want to export to a DELTA file.  

If you select a descriptor tree in combo box Restrict descriptors to tree: only that descriptor tree and its descriptors will be included in the analysis tree (see next section). Be aware that you might get warnings during the analysis step when the respective descriptor data are omitted.  

With option Include translations you allow the analysis step to write availble translations to the analysis tree. If you check this option and translations are available, you will see the available languages in the Export language: combo box after the analysis step.   

 

Analysis

After clicking the Analyse data button the data for export are collected in the database an included into an analysis tree in the lower part of the window. The analysis tree contains all data in a suitable format for the final export step. During data analysis the icon of the button changes to  and you may abort processing by clicking the button.

 

After analysis a message window inform you if any warnings or errors occured. If there are descriptions that reference descriptors that are not part of the export project, these descriptor data wil be omitted and the descriptions will be marked with yellow background (warning). When you move the mouse curser over the marked entries, you get information about the omitted descriptors as tool tip (see examples below). Anyway you may continue export or fix the problem by including the missing descriptors in the descriptor tree (see section "Editing the descriptor"). If errors are indicated, please save the error log file and send a feedback by pressing the button. 

 

Export

Before starting the export, the export file name should be checked. Either a single SDD rsp. DELTA file can be generated or output to several files may be chosen by selecting the Multi-file option. In case of SDD export currently the data will always be stored in a single file ("SDD1.xml") located in the directory specified in the "File name" text box.

If the "Multi-file" option for DELTA export is activated, the "File name" text box shows the output path, which is by default generated as <resources directory>\Export\<Database name>_<Project>. This default setting may be modified by editing the "File name" or by navigating at the target location by pressing the button. For DELTA export in the target directory the files "specs", "chars", "items" and "extra" will be generated. For SDD export you will get the file "SDD1.XML". For EML export you will get the files "EML_MetaData.xml" and "EML_DataTable.txt". 

If output to a single file is selected, by default the file name is generated as <resources directory>\Export\<Database name>_<Project>_<Type> and additionally the exension ".xml" (SDD export) or ".dat" (DELTA export) will be appended. <Type> is "SDD1" for SDD and "DELTA" for the DELTA format. This default setting may be changed by editing the "File name" or by navigating at the target location by pressing the button.

To start the export click on the Generate file button in the Write SDD group box to generate an SDD file or the Generate file button in the Write Delta group box to generate a DELTA file. Now the data from the analysis tree are written into the selected export file. The Comptible option controls generation of files with most possible compatibility to the DELTA rsp. SDD standard. On the other hand some data might not be present in the generated file, if this option is activated.

As an additional option you may generate file according the EML schema, which consists of a data table (tabulator separated text file) and an XML file that contains the metadata including column descriptions. Click on the Generate file button in the Write EML group box. The generated file names will have the endings "_EML_DataTable.txt" and "_EML_Metadata.xml".

Pressing the drop down button DELTA settings in the Write DELTA group box opens the DELTA writer options. You can chose to include some detail text and notes in the DELTA output (see left image below). For descriptions, descriptors or categorical states the details will be appended as DELTA comments (included in angle brackets "< ... >") to the respective titles. The notes will be appended as DELTA comments of the corresponding summary data. If you already generated DELTA files, the used settings will be automatically saved and you may restore them using the option Load last write settings. Finally click button DELTA settings to close the option panel. 

Pressing the drop down button EML settings in the Write EML group box opens the EML writer options. You can chose to include a special sign for empty column values or set the columns values in quotes (see center image below). Furthermore you may shose the column separator (tab stop rsp. comma) an decide if multiple categorical states shall be inserted as separate data columns. If you already generated EML files, the used settings will be automatically saved and you may restore them using the option Load last write settings. Finally click button EML settings to close the option panel. 

Pressing the drop down button SDD settings in the Write SDD group box opens the SDD writer options. You can chose to include translations in the SDD output (see right image above).  If you already generated SDD files ,the used settings will be automatically saved and you may restore them using the option Load last write settings. Finally click button SDD settings to close the option panel. 

For all three output formats you have the option Diversity Projects metadata. If this option is selected, additional information from the corresponding project in the linked projects and agents databases will be retrieved. An EML metadata file (XML) will be generated that includes the retrieved metadata and references the generated data file(s). 

While SDD can handle molecular sequence data, for DELTA export these data will be exported as text data. To preserve the sequence specific descriptor data, they will be inserted into the text character as a special comment with the format, e.g. "#6. Sequence descriptor <[SequenceCharacter][ST:N][SL:1][GS:-][/SequenceCharacter]>/". If during DELTA import this special comment is detected, it will be removed and the descriptor will be imported as a sequence descriptor using the given settings.

If the analysis tree includes sample data (see option Export sample data), they will be included as items at the end of the DELTA file. The naming of those spetial items will be <description name> - <event name> - Unit <number>. Sampling event data will not be included in the DELTA file.