Diversity Agents

Spreadsheets

The spreadsheets similar to the grids provide a tabular access to the data. In addition you can search and filter data independent from the main form. For an introduction see two short tutorials (overview), (tables and columns). To open a spreadsheet, choose Grid - Spreadsheets - and then one of the provided versions, e.g. Organisms from the menu. A window as shown below will open where the data are marked and organized according to the database tables. If values in one of the tables are missing, this will be indicated with a grey background. The colors correspond to the overview for the database.

Start

To use any of the spreadsheets as a starting window, click on the  button. Now the application will directly open the spreadsheet with the same parameters when you close it. 

Database

To change to another database, click on the button and choose the database as described here. Please keep in mind that the main form will connect to the new selected database. 

Project

If a user has read only projects available, a button will appear in front of the project. Click on this button to change to the list of projects with read only access. The window will change into the Read only mode (see below). To return to the list of projects with write access, just click no the button again. After changing the source for the project list, the project label will blink with red to remind you, to select a project from the list. 

Getting the data

The data are always restricted to one project, selected in the corresponding field (see image above) and restricted to the first top lines as specified in the Max. res. field. To change to the next or previous block of data, use the resp. button. 

The program will organize the data in blocks indicated by the colors as shown above. In the example above the first block contains 2  Organisms all belonging to the block starting with  Hildesheim... for the Event and M-0014196 for the Specimen. The next block then starts with  Hildesheim... indicated by a change in the color containing 2 Organisms. For most of the spreadsheets, the presence of the Event in the data is a prerequisit. As long as you do not sort or filter the data (see below), these blocks will be consistent with the data. As soon as you filter or sort the data, these blocks may be split and may not correspond to the data blocks in the database any more. In the example below you see the effect of sorting by the column Taxon on breaking up the blocks. E.g. the block for the specimen M-0036950 is split into 2.

SQL

The query used for the retrieval of the data can be accessed with a click on the SQL button.

Filtering

To filter the content of a table you can apply a filter either on the whole table or on each of the columns of the table.

Table filter

This filter will apply on the whole table. Click in the black box next to the filter area and select one of the filter options as shown below.

• ♦: Filled (all visible columns)
• ◊: Empty (all visible columns)
• •: Data in table do exist
• Ø: Data in table do not exist

Tutorial:

Column filter

Tutorial:

This filter will apply on a single column of a table. To set a filter click in the field underneath the column name. A window as shown below will open. With a click on the add button you can enter a value from the current content of the column as filter.

Here you set the sorting (see below), enter the search string and choose among several ways of comparison with the contents in the database:

• =: The content must be exactly like the given value
• ≠: The content must be different to the given value
• ~: The content must be similar to the given value, use wildcards % for any string or _ for a single character
• ¬: The content must NOT be similar to the given value, use wildcards % for any string or _ for a single character
• <: The content must be smaller than the given value
• >: The content must be bigger than the given value
• |: The content must be in a given list of values
• ∉: The content must NOT be in a given list of values

For columns linked to modules of the DiversityWorkbench there may be additional options. E.g. for a column linked to DiversityTaxonNames you get 3 additional options:

• +H: Include lower Hierarchy
• +S: Include Synonyms
• +H+S: Include lower Hierarchy and Synonyms

After selecting one of these options, you will be asked for the database and the project where the data should be taken from. In the next step the corresponding names as retrieved from the selected source will be listed (see below). To change the filter click on the button and on the delete button to remove it.

Sorting

↓↑ Tutorial:

The data can be sorted by each visible column. If you click into the field underneath the column name a window will open as described above. To change between the modes of sorting, just click on the current sorting. The modes are:

• -: Not sorted
• ↑: Sorted in ascending sequence
• ↓: Sorted in descending sequence

After all parameters are set, the sort mode and filter settings will be shown in the field underneath the column name (see image below). The number indicates the sequence within the sorting columns.

Timeout

After all parameters are set, click on the button. If for any reason you get a timeout, meaning the query for the data takes too long, either simplify your query or adapt the time for the query after a click on the button. If you do not want to restrict the time for the queries, set the value to 0. The default value is 30 seconds. 

Read only mode

<> Tutorial:

The formatting of the sheet including the color indication of tables and data blocks is rather time consuming. If these are not needed, e.g. for exporting the data, you can switch to the Read only mode for higher performance. For projects with read only access (see above), the mode is automatically set to Read only. 

Editing

To edit the data, just change them. If data are missing, type or select a value in the corresponding field. To remove data, click on the x field. This will remove all entries dependent on this entry that means all data right from the entry in the same line resp. block and any depending data. If there are any depending data, a window will open as shown below listing these data. Click OK if you want to remove the selected data including all depending data as shown in the window.

To change the content of several values in a column, either select the whole column with a click on the  button or select the fields manually. To avoid the pop-up of data-entry windows e.g. for columns with a lookup list, click no the button. Once you have selected the contents to change, select the mode of change:

• Prepend: Insert the given value at the beginning of the content
• Append: Append the given value at the end of the content
• Overwrite: Remove current content and insert given value
• Replace: Replace a string in the content with the given value
• Clear: Remove the content

After selecting the change mode, enter the text in the field where necessary and click on the corresponding button to perform the changes.

With the context menu (mark area and right click) you can Remove data (for single data fields with a list behind), Tranfer or Copy the content into the clipboard and Insert the content of the clipboard into the selected cells. Tutorial:

Locked columns

Columns that can not be edited are indicated with a lighter background color of the header and a gray text of title and content. These columns contain e.g. database generated content or data from lookup tables.

Inclusion of RowGUID

By default the primary key can not be changed in the spreadsheet. In some cases you may want to change parts of the primary key. To do this, you must include the column RowGUID in the list of selected columns. The program will ensure that the changed data do not collide with existing data, that is having the same primary key. See an example in a short tutorial:

Fixed sources

Tutorial:

For columns that are linked to a remote module, you can fix the source for the query. If the source is fixed, the query will directly contact the source as shown in the image below. To set resp. change the source, click on the button. A window will open where you can select either a webservice or a source database together with a project within the database. To remove the source, click on the button (only visible if a source is present).

To see an overview for the fixed sources, click on the button. A window will open where all sources are listed with the source for the current column marked with a yellow background. 

A "right-click" on the button in the main form will show the settings as well and with a click on the button you can set the connection for the selected column. If a column depends on values from the data as for example the relations to the module DiversityTaxonNames with a dependence on the taxonomic group, the relation will change with the selected row, depending on the data retrieved from the selected row. If there is no connection to the related module defined, the button will appear gray and if the column has no relation to a module, the button will appear like

If the source is not fixed the query will start as described here.

Settings

- To change the data shown in the query results, either click on the header of the tables, e.g. Event for that table. A window as shown below will open where you can select or deselect columns, change their headers etc. Red columns need to be filled (either by the database, the program or the user). In the Alias field you can rename the columns to your preferences. The button will show a description of the columns. For columns linked to a module, the current settings for the link will be shown in addition to the description. To display a column in the sheet use the checkbox . If a column is needed, but should be hidden, select the checkbox. Displayed columns are indicated by a yellow background while hidden columns get a light yellow background. Certain columns are required (indicated by a purple color) and can not be removed e.g. if they are involved in a link to a remote module like in the example below the columns FamilyCache and OrderCache. Columns linked to a module are indicated by a blue color.

The button allows you to include one of the tables missing in the sheet (see image below). After selection of the table, select the column(s) you want to include in the sheet as shown in the image above.

To store resp. load settings, use the resp. button. If a setting file has been corrupted click on the delete button to remove it and restart with factory settings. The spreadsheet will always start with the previous settings. The location where the settings are stored are set under Administration - Resources directory ... . For an introduction see a short tutorial .

Column width

Tutorial:

The width of the columns can be set manually (in the filter area) or automatically with a click on the button. A window will open where you can choose and set the parameters for setting the column width.

Adding

Tutorial:

To add new data, use the area underneath the data table. You can enter preset values that will be used for the new data. If the new data should be added to existing entries, select the corresponding row. Then click on the + field of the datarange that should be inserted. If preset values are given, all corresponding tables will be filled together with the new inserted data. Defaults for the responsibles can be set under Administration - Customize Display. If a column is linked to a remote module, the corresponding columns will be filled together with the selection of the link, so e.g. if you select a link for a taxonomic name to a DiversityTaxonNames the columns for the taxon and if available the familiy etc. will be filled as well. To remove a linked value, select it a choose Remove from the context menu (right click).

Export

Tutorial:

To export the data as a tab separated text file (UTF8), click on the button. You will be asked if you want to include the hidden columns. These are the primary keys you may need for statistic evaluations etc.

Feedback

To send a feedback to the developer of the software, create a screenshot and click on the  button. 

Details

Tutorial:

To see all details of a dataset, select the dataset in the sheet and click on the DWB button. A window, corresponding to the main form will open, showing all data related to the selected specimen. 

Image

Tutorial:

If an image is present a preview will be shown in the right upper area of the window. Click on it to see the image in a separate form. The images are restricted to the first image of one source, so e.g. in the organisms sheet only the first image of the specimen linked with this organism will be shown while images of e.g. linked to none or another organism or the collection event will not be addressed. 

Diversity Agents

Table Editors

For the data selected in the main window the table editors offer a direct access to the tables of the database. The menu Data - Table editors provide an editor for selected tables within the database.

A window with the content of the table will open. Columns with a gray background can not be edited here. Columns with a light gray background are linked to the contents of lookup tables where you can change according to the contents of these tables.

Select Set timeout ... from the menu to increase the default timeout from 5 seconds to a higher value, e.g. for greater amounts of data.  

Editing

You can either edit the contents of the table directly or perform changes to any number of marked fields. To mark a whole column use the  button. Once you have selected the contents to change, select one of the modes of change that appear in the upper left corner. The modes of change are:

• Insert: Insert the given value at the beginning of the content
• Append: Append the given value at the end of the content
• Replace: Replace a string in the content with the given value
• Clear: Remove the content

After selecting the change mode, enter the text in the field where necessary and click on the corresponding button to perform the changes

Deleting

If you select whole lines, a delete button will appear. Click on it to remove the selected datasets from the database.

Filtering

To filter the content of the table, click in the column that should be used for filtering. Then choose the mode of comparision:

• = : The content must be exactly like the given value
• ~ : The content must contain the given value
• ≠ : The content must be different to the given value

If you want the filtering to be case sensitive, choose the a<>A option. After all parameters are set, click on the filter button. To undo the filtering, click on the button. This will reset the data to the last saved version. If you want your changes to be saved, click the button before you reset the filtering. If you close the window all changes so far will be saved automatically. So if you do not want to save your changes, click on the button before closing the window.

Export

To export the data as a tab separated text file, click on the button. The file will be automatically saved in the client-folder.  

To export the data in a SQLite database click on the button. The data will be exported into the SQLite database DiversityAgentTables.sqlite in the folder Export in your application directory. If you want to save previous exports, please rename the SQLite database or copy it to a different directory.  

Log data

To see the log data of the table, you can click on the button. The content of the log table can not be changed, but is read only.  

Diversity Agents

Module Related Entry

The DiversityWorkbench is a set of components for building and managing biodiversity information. Each of which focuses on a particular domain and provides services for the other modules. For an introduction see a short tutorial .

To use the service of an external module you need access to the database of this module and optionally the module application placed in your application directory. Entries related to an external module have a standard interface in the main form.

Linking to a datasource or module

There are 4 options to link the content to a datasource (or module).

• Local database
• Remote database
• Cache database
• Webservice

There are two differnt approaches for providing content for the textbox. Either as a autocomplete list (, , ) or by searching values on base of the entered text in combination with a combobox ().

For the autocomplete lists just start to type and the corresponding entries will be listed where you can choose the entry you are looking for. To set the link including additional information provided by the original source (, ), you just have to leave the textbox. The program will then contact the database to which the entry corresponds and insert the additional information including the link to the source database.

The next paragaphs show how to uses this linking. The colours of the pins are automatically set when a datasource is set. The colour indicates if the current set datasource ist set to the local, remote or cache databse or to a webservice. To select one of the datasources, click on the  button and choose among the sources.

Local database

The values shown in the autocomplete list are taken from the entries in the local database.

This is the default. To refresh the content or slect the local databse as source, click on the pin button, choose the first empty entry of the list and click OK.

Remote database

To set the datasource on a remote database, choose a database from the list.

Either a database available via a linked server (e.g. [TNT.DIVERSITYWORKBENCH.DE,5432].DiversityTaxonNames_TaxaVaria see below) or a database on your local server, starting with e.g. DiversityTaxonNames for taxa in the identification

Next you will be asked to choose a project within the selected source. In case there are additional options you may be asked e.g. for taxa to restrict the content to a checklist provided by the datasource. The pin button and the textbox will change their color to orange to indicate the current source type. The autocomplete list is then generated based on the content in the remote database. When you leave the textbox the software will automatically add additional information from the selected source and set the link as shown below.

Cache database

To provide data via the cache database these data must be first imported as source as described in the link to cacheDB chapter.

This option provides faster access to remote data e.g. on linked servers . To choose a source from the cache database, click on the pin button and select the entry CacheDB from the list (see below) and click OK.

Next you will be asked to choose among the sources provided by the cache database. Choose your preferred source and click OK. The button and the textbox will change their color to to indicate the current source type. Now the values for the autocomplete list will be based on this datasource. Just start to type and the corresponding entries will be listed where you can choose the entry you are looking for. To set the link including additional information provided by the original source, you have to leave the textbox. The program will contact the database to which the entries in the cache database correspond and insert the additional information.

Webservice 

To set the datasource on a webservice, choose a webservice from the list as shown below. The pin button and the textbox will change their color to blue to indicate the current source type.

To get data from the webservice, you have to enter a search text in the textbox and click on the combobox button to search the webservice for matching entries. Choose among the provided results to fill the textfield and the set the link to the webservice.   

States and functions of the interface

There are several states and functions of this interface:

• 1: the value is only set in the local database with no connection to the remote module. In this state you may either type the value or select it from the values which are already available in the database (see above).

• 2: the value is related to the remote module.

If you wish to set a relation to the remote module you have 2 options:

Option 1

Click on the pin button to select a source as described above.

An overview for all settings can be found under Administration - Customize display - Settings.

Charts

After the source is set to a database of the DiversityWorkbench certain datasources provide also charts for the selection of taxa or terms. Click on the chart button to open a table where you can select one of the provided taxa resp. terms.

Option 2 Detailed linking with query option

Click on the amonite button. A window will open where you may select an entry from the foreign database. If the option for loading the connections is not set the connection to a database may not be established indicated by a button. Either requery all connections as described in chapter Connections or click on the button to open the database connections for the linked module. With this option you have the full range of query options as shown in the image below as well as access to additional webservices.

For DiversityScientificTerms you can use a hierarchy for the selection of the lists as shown below.

If the value has a relation to the remote module, the interface will appear as shown below with the content of textbox set to read only and a yellow background.

To release the connection to the remote module click on the button. If you require further information on the value, click on the DWB button. This will open a form showing an overview of the related value.

If the client application of the module is available, you may inspect the details of the entry. To start the client application of the remote module click on the DWB button. 

Removing defective links

If the provided link does not correspond to a dataset in the source, you will get a message as shown below. Click Yes to remove the wrong link and replace it with a valid link. 

Relation to a webservice

Some modules provide the possibility to link your data to an external webservice. For example, DiversityTaxonNames gives you access to the taxonomic names of IndexFungorum. To establish a connection to an external webservice click on the button. As with the link to modules within the DiversityWorkbench, a window will open where you are able to choose from either DiversityWorkbench modules or external Webservices. See chapter Webservice for further details

Diversity Agents

Display types

The type in which an agents name will be displayed can be set within a project.

To edit the display types, choose Administration - Agent display types ... from the menu. A window as shown below will open, where you can set the type for every project.

Every user has a default project within DiversityAgents. As an administrator, you can set the default projects for every user. The display type for a user is set according to these projects. To edit the default projects, choose Administration - Default projects ... from the menu. A window as shown below will open, where you can set the default project for every user. If the projects are not specified here, the first project the user has access to will be used.

Every user can set his default project by choosing Administration - Default project ... from the menu. A window as shown below will open, where the user can set the default project.

The tables used for the storage of these parameters are shown below.

Description of the tables

Table ProjectProxy

The projects - refers to database DiversityProjects

Depending on:

• AgentNameDisplayType_Enum

Table ProjectUser

The projects to which users have access to

Depending on:

• ProjectProxy
• UserProxy

Table UserProxy

The user logins - refers to database DiversityUsers

Depending on:

• ProjectProxy

Diversity Agents

Backup

The chapter below refers to the module DiversityCollection but is valid for DiversityAgents as well

If you want to create a backup of your database, there are 2 options. You may either export the data as csv files to your local computer or you may create a backup on the server.

Export data as csv

To export your data as csv files to your local computer, choose Data → Export → CSV(bcp) ... from the menu. A window will open as shown below, where you can select the tables that should be exported. Click on the Start Export button to export your data. If you choose the option as shown below 2 files will be created for every table. The first file (*.csv) contains the data while the second file (*.xml) contains the structure of the table.

Create backup on the server

To create a backup of your database on the server, choose Data → Backup database from the menu. This will create a SQL-Server backup on the server where the database is located. Ensure that there is enough space on the server.

Another option is to create a direct copy of the database files on the server. For this you have to use the functions provided by SQL-Server. However, you need administration rights for the database you want to create a backup of. Open the Enterprise Manager for SQL-Server, choose the database and detach it from the server as shown in the image below.

After detaching the database, you can save a copy of the ..._Data.MDF file to keep it as a backup.

After storing the backup you have to reattach the database.

A dialog will appear where you have to select the original database file in your directory.

Diversity Agents

Export CSV

The chapter below refers to the module DiversityCollection but is valid for DiversityAgents as well

To export the tables of the database in a tabulator, comma or semicolon separated format, choose Data → Export → Export CSV... from the menu. A window as shown below will open where you can select the tables to be exported in sections Selection criteria and in the Tables for export.

A prerequisite for this export is that the bcp program is installed on your computer. This has either been installed together with the installation of SQL-Server or you have to install the Microsoft Command Line Utilities for SQL Server.

To start the export click on the Start export button. By default the data will be exported into a directory ...\Export\<database_name> below your application directory. Click on the open button to select a different target directory before starting export.

After export the tables are marked with green background, if table schema and data were exported successfully. If only the data were exported, this is marked with yellow background, if nothing was exported, the background is red. A detailed export report can be viewed by a click on the export result file name.  

Diversity Agents

Export Wizard

The export wizard provides a possibility to export the data selected in the main form. The data are exported as tab separated text file. The export may include transformations of the data as well as information provided by linked modules and webservices. Choose Data - Export - Export wizard from the menu and then select one of the export targets (Event, Specimen, ...). For a short introduction see the tutorial.  

Adding tables

There are the following ways to add tables:

• One parallel table
• Several parallel tables according to selected data
• Dependent table

All options will include the depending tables as defined for the default table. The option for several tables will add as many tables as there are found in the data.

If you added parallel tables, you should set the sequence of the datasets within these tables: For the columns that should be used for sorting the data, set the ordering sequence to a value > 0 and choose if the ordering sequence should be ascending or descending .

Adding related values

Certain columns in the database may provide information linked to another table or a module resp. webservice . Click on the button to add a linked value.

Adding and editing file columns

To add columns to the exported file, use the buttons. In the textbox at the top of the file column, you can change the header for the column. To change the position of a file column use the resp. button. To fuse a column with the previous column, click in the gray bar on the left side of the column that will change to for fused columns. To remove a file column, use the delete button. Pre- and postfixes for the columns can directly be entered in the corresponding fields. To apply transformations on the data click on the button.  

Filter

To filter the exported data, use the filter function. Click on the button and enter the text for the filter. Only data matching the filter string will be exported. If a filter is set, the button will have a red background to remind you of the filter. The filter may be set for any number of columns you need for the restriction of the exported data.  

Rowfilter

This filter in contrast to the filter above strictly applies to the row according to the sequence of the data. For an explanation see a short tutorial .

Test

To test the export choose the Test tab, set the number of lines that should be included in the test and click on the Test export button. To inspect the result in a separate window, click on the button.

SQL

If you want to inspect the SQL commands created during the test check this option. To see the generated SQL click on the SQL button after the Test export. A window containing all commands including their corresponding tables will be shown.

Export

To export your data to a file, choose the Export tab. If you want to store the file in different place use the button to choose the directory and edit the name of the file if necessary. Check the include a schema option if you want to save a schema together with your export. To start the export, click on the Export data   button. To open the exported file, use the button.

Export to SQLite

To export your data into a SQLite database, choose the Export to SQLite tab. You may change the preset name of the database in order to keep previous exports. Otherwise you overwrite previous exports with the same filename. To start the export, click on the Export data   button. To view the exported data, use the button.

Schema

To handle the settings of your export, choose the Schema tab. To load a predefined schema, click on the open button. To reset the settings to the default, click on the undo button. To save the current schema click on the save button. With the button you can inspect the schema in a separate window.

Diversity Agents

Import wizard

The import wizard is the general way to import data into a Diversity Workbench module database. It allows the import from tab separated text files (tsv) into the database tables. The key import steps, the definition of a mapping from tab seperated columns in the text file to the coresponding database table columns, is designed in the import wizard. As the mapping can be sometimes cumbersom to develop, the import wizard allows to save the mapping for repeated import of equaly structured tsv files.

The examples below are from the module DiversityAgents, but are valid for any other module as well.

With the current solution please ensure that there are no concurrent imports in the same database.

With this import routine, you can import data from text files (as tab-separated lists) into the database. A short introduction is provided in a video . Choose Data → Import → Wizard → Agent from the menu. A window as shown below will open that will lead you through the import of the data. The window is separated in 3 areas. On the left side, you see a list of possible data related import steps according to the type of data you choose for the import. On the right side you see the list of currently selected import steps. In the middle part the details of the selected import steps are shown.

Choosing the File and Settings

• File: As a first step, choose the File from where the data should be imported. The currently supported format is tab-separated text. Choosing a file will automatically set the default directory for the import files. To avoid setting this directory, deselect the option Adapt default directory in the context menu of the button to open the file.
• Encoding: Choose the Encoding of the file, e.g. Unicode. The preferred encoding is UTF8.
  Encoding changed

  Starting with version 4.6 the default encoding changed to UTF8. Please use Windows for old ANSI encodings. The program will try to detect the encoding and warn you if it does not match the current encoding. The detection of the encoding depends on the presence of a BOM and does not work for every file, e.g. ANSI-encoding

• Lines: The Start line and End line will automatically be set according to your data. You may change these to restrict the data lines that should be imported. The not imported parts in the file are indicated as shown below with a gray background. If the
• First line: The option First line contains the column definition decides if this line will not be imported.
• Duplicates: To avoid duplicate imports you can Use the default duplicate check - see a video for an explanation.
• Language: If your data contains e.g. date information where notations differ between countries (e.g. 31.4.2013 - 4.31.2013), choose the Language / Country to ensure a correct interpretation of your data.
• Line break: With the option Translate \r\n to line break the character sequence \r\n in the data will be translated in a line break in the database.
• SQL statements: To save all SQL statements that are generated during a test or import, you can check the option Record all SQL statements.
• Schema: Finally you can select a prepared Schema (see chapter Schema below) for the import.

Choosing the data ranges

In the selection list on the left side of the window (see below) all possible import steps for the data are listed according to the type of data you want to import.

The import of certain tables can be paralleled. To add parallels click on the add button (see below). To remove parallels, use the button. Only selected ranges will appear in the list of the steps on the right (see below).

To import information of logging columns like who created and changed the data, click on the include logging columns button in the header line. This will include additional substeps for every step containing the logging columns (see below). If you do not import these data, they will be automatically filled by default values like the current time and user.

Attaching data

You can either import your data as new data or Attach them to data in the database. Select the import step Attachment from the list. All tables that are selected and contain columns at which you can attach data are listed (see below). Either choose the first option Import as new data or one of the columns the attachment columns offered like SeriesCode in the table Series in the example below.

If you select a column for attachment, this column will be marked with a blue background (see below and chapter Table data).

Merging data

You can either import your data as new data or Merge them with data in the database. Select the import step Merge from the list. For every table you can choose between Insert, Merge, Update and Attach (see below).

The Insert option will import the data from the file independent of existing data in the database.

The Merge option will compare the data from the file with those in the database according to the Key columns (see below). If no matching data are found in the database, the data from the file will be imported. Otherwise the data will be updated.

The Update option will compare the data from the file with those in the database according to the Key columns. Only matching data found in the database will be updated.

The Attach option will compare the data from the file with those in the database according to the Key columns. The found data will not be changed, but used as a reference data in depending tables. 

Empty content will be ignored e.g. for the Merge or Update option. To remove content you have to enter the value NULL. As long as the column will allow emty values, the content will be removed using the NULL value.

Table data

To set the source for the columns in the file, select the step of a table listed underneath the Merge step. All columns available for importing data will be listed in the central part of the window. In the example shown below, the first column is used to attach the new data to data in the database.

A reminder in the header line will show you which actions are still needed to import the data into the table:

• Please select at least one column   = No column has been selected so far.
• Please select at least one decisive column   = If data will be imported depends on the content of decisive columns, so at least one must be selected.
• Please select the position in the file   = The position in the file must be given if the data for a column should be taken from the file.
• Please select at least one column for comparison   = For all merge types other than insert columns for comparison with data in the database are needed.
• From file or For all   = For every you have to decide whether the data are taken from the file or a value is entered for all
• Please select a value from the list   = You have to select a value from the provided list
• Please enter a value   = You have to enter a value used for all datasets

The handling of the columns in described in the chapter columns.

Testing

- To test if all requirements for the import are met use the Testing step. You can use a certain line in the file for your test and then click on the Test data in line:  button. If there are still unmet requirements, these will be listed in a window as shown below.

If finally all requirements are met, the testing function will try to write the data into the database and display any errors that occurred as shown below. All datasets marked with a red background, produced some error.  

To see the list of all errors, double click in the error list window in the header line (see below).

If finally no errors are left, your data are ready for import. The colors in the table nodes in the tree indicate the handling of the datasets:

• INSERT
• MERGE
• UPDATE,
• No difference
• Attach
• No data

The colors of the table columns indicate whether a column is decisive , a key column or an attachment column .  

If you suspect, that the import file contains data already present in the database, you may test this and extract only the missing lines in a new file. Choose the attachment column (see chapter Attaching data) and click on the button Check for already present data. The data already present in the database will be marked red (see below). Click on the button Save missing data as text file to store the data not present in the database in a new file for the import. The import of agents contains the option Use default duplicate check for AgentName that is selected by default. To ensure the employment of this option the column AgentName must be filled according to the generation of the name by the insert trigger of the table Agent (InheritedNamePrefix + ' ' + Inheritedname + ', ' + GivenName  + ' ' + GivenNamePostfix + ', ' + InheritedNamePostfix + ', ' + AgentTitle - for details, see the documentation of the database).

If you happen to get a file with a content as shown below, you may have seleted the wrong encoding or the encoding is incompatible. Please try to save the original file as UTF8 and select this encoding for the import. 

Import

- With the last step you can finally start to import the data into the database. If you want to repeat the import with the same settings and data of the same structure, you can save a schema of the current settings (see below). You optionally can include a description of your schema and with the button you can generate a file containing only the description.

Schedule for import of tab-separated text files into DiversityAgents

• Target within DiversityAgents: Agent
• Database version: 02.01.13
• Schedule version: 1
• Use default duplicate check: ✔
• Lines: 2 - 7
• First line contains column definition: ✔
• Encoding: UTF8
• Language: US

Lines that could not be imported will be marked with a red background while imported lines are marked green (see below).

If you want to save lines that produce errors during the import in a separate file, use the Save failed lines option. The protocol of the import will contain all settings according to the used schema and an overview containing the number of inserted, updated, unchanged and failed lines (see below).

Description

- A description of the schema may be included in the schema itself or with a click on the Import button generated as a separate file. This file will be located in a separate directory Description to avoid confusion with import schemas. An example for a description file is shown below, containing common settings, the treatment of the file columns and interface settings as defined in the schema.

Administration

• How to set the access to projects
  • see chapter Zugriff auf Projekte
  • and the video : Einstellungen des Zugriffs auf Projekte, Zugriff auf Daten innerhalb von Projekten

Editing

• How to set the language of a dataset
  • see chapter Sprache
  • and the video : Festlegen der Sprache des Inhalts eines Datensatzes.
  • How are addresses retrieved from the hierarchy
    • see chapter Hierarchien
    • and the video : Haupthierarchie mit Ermittlung der Adresse und zusätzliche Hierarchien.
      • How to remove loops from a hierarchy
        • see chapter Schleifen entfernen
        • and the video : Entfernen von Schleifen in der Hierarchie anhand von Beispieldaten.
  • How to remove loops in the synonymy
    • see chapter Synonyme: Schleifen entfernen
    • and the video : Entfernen von Schleifen bei Synonymen anhand von Beispieldaten.
  • How to see the public contanct infos
    • see chapter Öffentliche Kontaktdaten
    • and the video : Ermittlung der öffentlichen Kontaktdaten anhand zugänglichen Kontaktdaten des aktuellen Eintrags sowie der in der Haupthierarchie verfügbaren Kontaktdaten.
  • How to set the icons of descriptors
    • see chapter Descriptoren - Icons einstellen
    • and the video : 2 prinzipielle Typen von Deskriptor: Modul zugeordnet oder ohne Modul. 4 Möglichkeiten für die Einstellung der Icons - OHNE ICON: Modul zugeordnet: Default Icon des Moduls. ohne Modul: Default Icon für Deskriptor. MIT ICON: ohne Modul: Ausgabe des gesetzten Icons. Modul zugeordnet: Ausgabe des gesetzten Icons. Default Icon des Moduls wird ersetzt.
  • How to propagate changes to linked data
    • see chapter Änderungen verlinkter Datensätze weiterleiten
    • and the video : DiversityAgents als Modul in der DWB. Darstellung der Beispieldaten in DiversityCollection und DiversityAgents. Änderung in DiversityAgents, Eintrag des geänderten Datensatzes in Update Liste. Auswahl der zu aktualsierenden Datenbanken. Display types in DiversityAgents. Eintrag von aktuellem Datensatz, Eintrag aller Datensätze, Ansicht der Liste, Löschen der Liste. Aktualisieren des Datensatzes. Kontrolle der Änderungen in DiversityCollection.

  • Spreadsheet

• How to include column RowGUID to enable changes in the PK of a table
  • see the video RowGUID einschliessen : Erläuterung der RowGUID. Beispieldaten im TableEditor. Laden der Daten im Spreadsheet. Versuch der Änderung einer Spalte die Teil des Primärschlüssels ist. Einschliessen der RowGUID. Versuch der Änderung mit eingeschlossener RowGUID. Ablehnung der Änderung bei Kollision mit Primärschlüssel. Änderung der Daten ohne Kollision. Ergebnis der Bearbeitung.

Import

• How to import data
  • see chapter Überblick
  • and the video : Import von Daten anhand des Beispiels im Tutorial.
  • How to check for duplicates
    • see chapter Duplikate
    • and the video : Prüfung auf in der Datenbank bereits vorhandene Duplikate während des Imports.
  • How to transform data during the import
    • see chapter Transformation
    • and the video . Transformation von Daten für den Import am Beispiel eines Datums.

Archive

• How to create an achive including the log tables
• see chapter Archiv inclusive des Logs
• and the video .
  (Aufbau von Logtabellen, Vorstellung der Beispieldaten, Erstellung eines Archivs: Einschliessen der Logtabellen, Suche nach Daten, Anlegen des Archivs. Wechsel in leere Datenbank. Einlesen der Archivdaten: Auswahl des Archivs, Log einschliessen, Einlesen der Daten, Archivdaten einspielen, Meldung zu vorhandenem Projekt. Anmeldebestätigung. Ansicht der Daten einschliesslich der Daten in den Logtabellen)
• How to create an archive
• see chapter Archiv erstellen
• and the video .
  (Automatisierte Erstellung durch Server, Manuelle Erstellung, Suche nach Daten, Erstellung, Inhalt einer xml-Datei, Protokoll).

Diversity Agents

General

• Customize

  Customizing the user interface

• Feedback

  Sending feedback for the software

Diversity Agents

Internals

For users that need more detailed information about the software including database design etc.