Diversity Workbench

For an overview of the modules of the DiversityWorkbench see chapter DiversityWorkbench

Subsections of Diversity Workbench

DiversityWorkbench

The chapters below describe features that are available in several modules of the Diversity Workbench resp. provided via a shared library like DiversityWorkbench.dll

Upcoming version

4.2.13

  • Export wizard
    • Bugfix searching for parent of start table
  • Query
    • Bugfix when QueryMainTableLocal is null

Current version

4.2.12

  • Import wizard
    • Bugfixes for duplicate check if no data are available
  • Export wizard
    • Bugfixes if no data are available in parent table
    • Showing generated SQL commands

Previous versions

4.2.11

  • Import wizard
    • Bugfixes for duplicate check for data containing ’ in values of checked column
    • Bugfixes for column pre- and postfix containing ’ in values
    • Option to prevent insert into column
    • Description for columns expanded and redesigned
  • Bugfixes setting linked servers
  • JsonCache: New interface for the modules containing a json summary for datasets

4.2.10

  • Webservice Catalogue of Life available
  • Several bugfixes

4.2.9

  • UserControlQueryList: Dropdownwidth for column selection set to 200 to ensure readablility

4.2.8 (2023-08-30)

  • UserControlQueryList - bugfix if for readonly no valid boolean expression is returned
  • Documentation
    • Bugfix - Listing objects
    • Bugfix - Setting title
    • Bugfix - Selecting schema
    • Bugfix init HUGO replacements

4.2.7 (2023-08-17)

  • Formfunctions
    • Bugfix missing command in function public bool initSqlAdapter(ref System.Data.SqlClient.SqlDataAdapter Adapter, string SQL, System.Data.DataTable Table)
    • Bugfix WorkbenchUnit in function public static string getDatabaseNameFromURI(string URI)
    • Bugfix Project.cs - using ViewBaseURL instead of function or linked server
  • Archive
    • Optional inclusion of log tables
    • Bugfixes resetting database
    • Several bugfixes getting data
    • Several bugfixes restoring archive
    • Progress bars included
    • Extension for setting specific query to get IDs for temp table

4.2.6 (2023-08-01)

  • Documentation
    • Bugfix listing objects
    • HUGO
      • Export of enumeration table content
  • Updating depending datasets Video starten.
    • Context menu:
      • Adding current dataset.
      • Adding all datasets from the list.
      • Showing content of the list.
      • Clearing the list.
  • Bugfix setting backlinks for DiversityCollection linked to DiversityAgents
  • Bugfix setting SQL for backlink updates
  • Bugfix setting backlinks for DiversityProjects linked to DiversityAgents
  • Bugfix setting images in tree of FormEnumAdministration
  • FormFunctions: public static string IsoDate(string Text, bool SpaceAsSeparator = true, bool RestrictToDate = false) expanded with optional restriction to date

4.2.5 (2023-07-25)

  • TableEditor:
    • Bugfix in initForm
  • Enumeration administration:
    • Table editor included
  • Agent - Bugfix getting synonyms in unit values
  • Formfunctions - Autocompletion - Bugfix missing Alias for table
  • Documentation
    • HUGO
      • Export as markdown for HUGO included
      • Replacements and adaptions for HUGO included
  • Spreadsheet:
    • Optional inclusion of RowGUID for tables where PK should be edited Video starten
    • Hiding RowGUID by default

4.2.4 (2023-07-12)

  • FormFunctions:
    • public static string IsoDate(string Text, bool SpaceAsSeparator = true)
  • Documentation:
    • Default output set to HUGO
    • HUGO output
      • Index removed
      • Application objects removed
      • ER-diagram located underneath table header
      • Bugfix ER-diagram path
      • Bugfix Enum output
      • Include menu icon

WpfControls

Current version

1.0.1

  • Bugfix rescanning image if values for height and width are missing
  • Showing message for button ShowDetail if no detail is defined

Installation

Resources

To run a module of the Diversity Workbench you need the database and the client. All parts can be downloaded for free from http://www.microsoft.com/downloads/ and http://www.diversityworkbench.net/Portal/.

For an intoduction see a short tutorial with DiversityCollection as an example .

Client

The client is based on the .Net framework version 4.8 from Microsoft. If not already present, the software will ask you for installation. Download the installer from the Portal and start it. In case you get as message as shown below

Please click on Weitere Informationen. A button as shown below will appear

Please click on Trotzdem ausführen to install the software as shown below.

If you need to change the location of the software, this can be done in the next step (see below).

After the installation is done, the software will be included in the programm menu (see below) and a shortcut on the desctop will be created.

The software will be placed in the programs directory as shown below.

After the installation make sure to get the latest updates from http://windowsupdate.microsoft.com/.

 

Run program without installer

There are several reasons why you might prefer to run DiversityCollection without installation. E.g. if you lack administrative permissions on your computer or if you want to use several different versions of DiversityCollection in parallel. If the downloaded .zip file contains besides the installer DiversityCollectionSetup.msi the file unpack.bat, you can easlily unpack the program files to a directory on your desktop. Unpack the .zip file to a local directory and start the batch file unpack.bat with a double-click. You might get a security warning as shown in the section above, see there how to proceed. The batch file unpacks the program files to a folder on your desktop named DiversityCollection_x_x_x, where "x_x_x" stands for the program version. In the folder double-click on file DiversityCollection.exe to start the program

If the downloaded .zip file does not contain the file unpack.bat you can extract the files manually. Extract the content of the  .zip file, open the cmd and use msiexex to extract the files to the desired location as shown in the example below:

msiexex /a c:\\Downloads\\DiversityCollectionSetup.msi /qb TARGETDIR=c:\\DiversityWorkbench\\DiversityCollection

where  c:\\Downloads is the directory containing the msi file and c:\\DiversityWorkbench\\DiversityCollection is your target directory. Now you can start the program with a double click on DiversityCollection.exe located in your specified directory.

 

Database

For the installation of a local database see the section Installation of the database.

 

Display / Monitor settings

By default the program detects monitor resolution higher than 100% and will adapt to it. The icons of the windows may become somewhat blurry. If for any reason the whole program stays blurry it may help to let windows try to fix it (see image below).

 

Database login

Connect to a database

To access the database select from the menu. A dialog will open as shown below. If you have been connected to a database in previous sessions you can select on of these session with the option. Otherwise you can specify the name or IP-adress. For a test you can send a ping with a click on the button (see below).

Next enter the port number of the server and the authentication mode

  • Windows authentication: Using the Windows Login information.
  • SQL-Server authentication: User and password as defined in the database.

  

The standard port number for SQL-Server is 1433 and will be set as a default (see left image above). If the database server is configured using a port different from that port, you must give the port number in the field Port (see left image above). If the connection informations are valid, you can choose a database from the server. In the main form you can select the databases that are available under Connection - Database... Video starten

If you access a database for the first time you will be asked to consent to the storage and processing of your personal data (see below) according to the General Data Protection Regulation. Without your consent the access is not possible.

Encryption

The connection to the databases by default is encrypted, symbolized by the icon. To change to an not encrypted connection, click on the icon which will change to accordingly, indicating a not encrypted connection.

Training

For the first steps we provide a training installation on:

training.diversityworkbench.de,5432

Please turn to the development team for a login.

To connect to the database please follow the instructions in chapter Connect to a database.

Every module contains a tutorial for the first steps.

Query

Visibility

The query containing results and fields is positioned a the left of the window. To visibility of the query can be set in the menu: Query - Show query:

The query fields are displayed in the main form beneath the list of the result items. You can change this arrangement using the  /  button to place the query fields on the left side of the item list.

To hide the area containing the search fields click on the  button. If the search area is hidden and you want to start a new search, just click on the  button.

Searching

To search for data, enter the restrictions in the fields for the search conditions and click on the  button. The agents found in the database will be shown in the list. To add agents with differing search conditions click on the  button. If you want to remove entries from the selected list, choose them and click on the  button. This will not delete the data from the database, but remove them from your query result.

The option to provide a result list with many columns is described in chapter Many result columns.

To change the displayed fields for searching data click on the  button to change the query options

A window will open containing the command used to retrieve the data (see below).

To keep the values you entered for the query, you can click on the button. It will change to and when restarting the program your query values will be remembered

To speed up the query you may use the optimized version (Standard query: Optimized query: ). The optimized query in combination with the remembering option includes the functionality to preselect data depending on the query restrictions, e.g. if you search for an identification and enter a taxonomic name the first identification matching the restrictions will be preselected in the main window as shown below.

The set the option for optimized queries and the option to remember the query settings as default, choose Administration - Customize display... from the menu. In the tab Defaults and miscellaneous select the Optimized and Remember option. The buttons in the query will be hidden providing more space. 

Certain query fields provide the option to add up to 3 duplicates of themselves. Use the button to add and the button to remove duplicates. The restrictions can be combined with AND + and OR |. Just click on the + resp. | sign to change between the modes.

Within the query options you have several possibilities to specify your search restriction. Use the drop-down menu to switch between operators. The available operators are shown in the tables below.
Availability: t = text, n = numeric, d = date, h = hierarchy, x = XML, e = EXIF, g = geography, a = annotation, m = module
For an introduction see two short tutorials

Query operators

Op. Meaning Example Avail.
search for an entry like… (you may use wildcards) Pinus s[iy]lvestris % tn---e--
= search for an entry exactly equal to … Pinus silvestris L. tndh----
search for an entry not like... Pinus s[iy]lvestris % (you may use wildcards) tn------
search for an entry where a value is missing … tndhxeg-
search for an entry where a value is present … tndhxeg-
- search for an entry between … and … 2000 - 2003 tnd-----
| search within a list of entries, separated by "|" or one line per value. If wildcards ("%", "_") are included the query will search for values like the given values 2000 | 2003 ... | 2000 2003 tn------
search with exclusion of a list of entries with one value per line. Wildcards will be ignored 2000 2003 tn—–m
search with exclusion of a list of entries with one value per line. Wildcards will be ignored 2000 2003 tn—–m
± search for an entry where only parts of the date are present… -.-.2006 --d-----
Δ search including children in a hierarchy … M-Fungi ---h----
/ Search for entries containing a given XML node (not for EXIF) settings ----x---
¬ Search for entries not containing a given XML node (not for EXIF) settings ----x---
Search for entries with a maximal distance of … POINT(24.24 45.243) | 50 km ------g-
O Search for entries within an area POLYGON(34.5 ... ------g-
¤ Search for entries outside an area POLYGON(34.5 ... ------g-
+H Search for entry including lower hierarchy Picea | Picea abies -------m
+S Search for entry including synonyms Picea abies | Pinus abies -------m
+HS Search for entry including lower hierarchy and symonyms Picea | Picea abies | Pinus abies -------m

Operators for text

Operator Meaning Example
search for an entry like ... Pinus s[iy]lvestris % (you may use wildcards)
= search for an entry exactly equal to ... Pinus silvestris L.
search for an entry not like ... Pinus s[iy]lvestris% (you may use wildcards)
search for an entry where a value is missing ...
- search for an entry between ... and ... 2000 - 2003
| search within a list of entries, separated by "|" or one line per value. If wildcards ("%", "_") are included the query will search for values like the given values 2000 | 2003 ... | 2000 2003
search with exclusion of a list of entries with one value per line. Wildcards will be ignored 2000 2003
search with exclusion of a list of entries with one value per line. Wildcards will be ignored 2000 2003

Operators for numeric data

Operator Meaning Example
= search for an entry exactly equal to … Pinus silvestris L.
< search for an entry lower than ... 2006
> search for an entry higher than ... 2006
- search for an entry between ... and ... 2000 - 2003
| search within a list of entries, separated by "|" or one line per value 2000 | 2003
search within a list of entries with one value per line 2000 2003
search with exclusion of a list of entries with one value per line 2000 2003
search for an entry where a value is missing ...
search for an entry where a value is present

Operators for date values

Operator Meaning Example
= search for an entry exactly equal to ... 20.3.2006
< search for an entry lower than ... 20.3.2006
> search for an entry higher than ... 20.3.2006
- search for an entry between two dates... 20.3.2006 - 29.3.2006
± search for an entry where only parts of the date are present... -.-.2006
search for an entry where the date is missing …
search for an entry where the date is present and complete …

Oparators for hierachy

Operator Meaning Example
= search for an entry exactly equal to ... M-Fungi
search for an entry which is not equal to … M-Fungi
search for missing entry ...
search for present entry ...
Δ search including children in a hierarchy ... M-Fungi

Operators for XML

Operator Meaning Example
~ Search for entries like a given text (for EXIF) RDF
/ Search for entries containing a given XML node (not for EXIF) settings
¬ Search for entries not containing agiven XML node (not for EXIF) settings
search for missing entry ...
search for present entry ...

Operators for geography

Use (via local files or manual creation) resp. (via DiversityGazetteer) button to set geography

Operator Meaning Example
Search for entries with a maximal distance of ... POINT(24.24 45.243) | 50 km
O Search for entries within an area POLYGON(34.5 ...
¤ Search for entries outside an area POLYGON(34.5 ...
search for missing entry...
search for present entry ...

For yes/no fields exists a checkbox with 3 options: = yes, = no, = undefined

For a search within a lists (using the |, ∈ or ∉ operator) you may double click in the text field to open a window where you can enter your values. To separate the values either use the "|" sign or a new line as shown in the image below.

Query modules

In the query for values linked to another module click on the button to open a window where you can search for values within the linked module. A click on the button will show a list of the selected items. For some modules you can choose the cache database as source (if you have the proper permissions). This option provides a faster query esp. if e.g. the data of a related module are accessed via a linked server.

Tutorial:

Operator Meaning Example
Search for entries with a list Rosa | Rosa caninia \
Search for entries not within a list Rosa | Rosa caninia | ...
+H search for entry including lower ierarchy Picea | Picea abies | ...
+S search for entry including synonyms Picea abies | Pinus abies | ...
+HS search for entry including lower hierarchy and synonyms Picea | Picea abies | Pinus abies | ...
Change filter mode between link and text http://tnt.diversityworkbench.de/TaxonNames_Plants/4269 <> Picea abies L.

Subsections of Query

Query - many result columns

The option to include several columns in the result list is only available for optimized queries . To display several columns in the result list, click on the button next to the order by column (see below). A window will open where you can select the next column for the sorting. Underneath the order by column the second column for the sorting will be added. To remove this sorting column you can click on the    button. By default the width for the columns is set to 10. You can change this according to content. To set the width to the maximal length of the current content, click on the button. To change the sorting of the added order column click on the button. The colums are separated via " | " as shown in the image below.

 

Query options

To change the displayed fields for searching agents click on the  button. This opens a form where you can select and deselect the fields you want to be shown for searching agents. Here you can change the maximal numbers of items that well be shown in the list. If you want certain fields to appear in the query options select of deselect them in the list.

The limit for the creation of drop down lists is by default set to 1000. If you connect to a database, the programm will create drop down list based on the entries in the database to ease the entry in the query. For slow connections you may set this to lower value to speed up the start of the program. If you set the value to 0 no drop down lists will be created.

After editing the query options click OK so the new selection will become active for the next query.

The maximal number of items shown in a query result can be set in the form for the query options. The default value is set to 100. have If you need to see more or less results, you may change this value to any number you like (for higher numbers make sure you have fast connection to your database).

Wildcards

Wildcards in SQL

There are 4 different possibilities for wildcards in SQL:

Operator Description Example Result
%   any string consisting of no, one or many characters Pinus % will find anything like Pinus, Pinus sylvestris, Pinus strobus etc.
*  same effect as % (see above)
_  a single character Pinus s_lvestris will find Pinus sylvestris and Pinus silvestris etc.
[…] any character out of a given range like [abcde] or [a-e] Pinus s[iy]lvestris will find Pinus sylvestris and Pinus silvestris
[^…]   any character not in a given range like [^abcde] or [^a-e] Pinus s[^i]lvestris will find Pinus sylvestris but not Pinus silvestris

Editing the data

The main forms of all modules have a design as shown below

Main form Main form

Status

In the status area you find

  • The name of the module,
  • The name of the database if different fron the module
  • The version of the software

The menu contains the options provided by the module

Commands

The commands area contains buttons for e.g. handling the data:

  • - To connect to a database click on the button.
  • - To save the changes in a dataset click on the button.
  • - To propagate the changes in a dataset to linked modules click on the button.
  • - To undo the changes a dataset click the button. This will recover the original data unless the changes had been saved or changes were done in certain tables or hierarchies were the data must be stored to display the hierarchy.
  • - To create a new entry in the database, click on the  button below the search result listbox. This will create a entry with the specimen and show it in the list.
  • - To copy a specimen, choose it from the list and click on the button .
  • - To options a dataset click on the button .
  • - To change the arrangement of the query click on the button.

Subsections of Editing the data

Module Related Entry

The DiversityWorkbench is a set of components for building and managing biodiversity information. Each of which focuses on a particular domain and provides services for the other modules. For an introduction see a short tutorial Video starten. To use the service of a module you need access to the database of the module and optionally the module application placed in your application directory. Entries related to an external module have a standard interface in the main form.

Linking to a datasource

There are 4 options to link the content to a datasource.

  • Local database
  • Remote database
  • Cache database
  • Webservice

There are 2 versions for providing content for the textbox. Either a autocomplete list (, , ) or searching values on base of the entered text in combination with a combobox (). For the autocomplete list just start to type and the corresponding entries will be listed where you can choose the entry you are looking for. To set the link including additional information provided by the original source (, ), you have to leave the textbox. The program will contact the database to which the entry corresponds and insert the additional information.  

Local database

The values shown in the autocomplete list are taken from the entries in the local database.

This is the default. To refresh the content, click on the button, choose the first empty entry and click OK. To select one of the datasources below, click on the  button and choose among the sources (see below).

Remote database

To set the datasource on a remote database, choose a database from the list.

Either a database available via a linked server (e.g. [TNT.DIVERSITYWORKBENCH.DE,5432].DiversityTaxonNames_TaxaVaria see below) or a database on your local server, starting with e.g. DiversityTaxonNames for taxa in the identification

Next you will be asked to choose a project within the selected source. In case there are additional options you may be asked e.g. for taxa to restrict the content to a checklist provided by the datasource. The button and the textbox will change their color to to indicate the current source. The autocomplete list is then generated based on the content in the remote database. As with the cache database as source, you have to leave the textbox to add additional information from the selected source and set the link as shown below.

 

Cache database

To provide data via the cache database these data must be imported as source as described here.

This option provides faster access to remote data e.g. on linked servers . To choose a source from the cache database, click on the button and select the entry CacheDB from the list (see below) and click OK.

Next you will be asked to choose among the sources provided by the cache database. Choose your preferred source and click OK. The button and the textbox will change their color to to indicate the current source. Now the values for the autocomplete list will be based on this datasource. Just start to type and the corresponding entries will be listed where you can choose the entry you are looking for. To set the link including additional information provided by the original source, you have to leave the textbox. The program will contact the database to which the entries in the cache database correspond and insert the additional information.

Webservice 

To set the datasource on a webservice, choose a webservice from the list as shown below. The button and the textbox will change their color to to indicate the current source.

To get data from the webservice, you have to enter a search text in the textbox and click on the combobox to search the webservice for matching entries. Choose among the provided results to set the link to the webservice.   

States and functions of the interface

There are several states and functions of this interface:

  • 1: the value is only set in the local database with no connection to the remote module. In this state you may either type the value or select it from the values which are already available in the database (see above).

  • 2: the value is related to the remote module.

If you wish to set a relation to the remote module you have 2 options:

Option 1

Click on the button to select a source as described above.

An overview for all settings can be found under Administration - Customize display - Settings.

After the source is set to a database of the DiversityWorkbench certain datasources provide charts for the selection of taxa or terms. Click on the button to open a table where you can select one of the provided taxa resp. terms. Video starten

 

Option 2

Click on the button. A window will open where you may select an entry from the foreign database. If the option for loading the connections is not set the connection to a database may not be established indicated by a button. Either requery all connections as described in chapter Connections or click on the button to open the database connections for the linked module. With this option you have the full range of query options as shown in the image below as well as access to additional webservices.

For DiversityScientificTerms you can use a hierarchy for the selection of the lists as shown below.

 

If the value has a relation to the remote module, the interface will appear as shown below with the content of textbox set to read only and a yellow background.

To release the connection to the remote module click on the button. If you require further information on the value, click on the button. This will open a form showing an overview of the related value.

If the client application of the module is available, you may inspect the details of the entry. To start the client application of the remote module click on the button. 

If the provided link does not correspond to a dataset in the source, you will get a message as shown below. Click Yes to remove the wrong link and replace it with a valid link. Video starten

 

Relation to a webservice

Some modules provide the possibility to link your data to an external webservice. For example, DiversityTaxonNames gives you access to the taxonomic names of IndexFungorum. To establish a connection to an external webservice click on the button. As with the link to modules within the DiversityWorkbench, a window will open where you are able to choose from either DiversityWorkbench modules or external Webservices. See Webservice for further details

Data in tables

Besides of the main form, the Diversity Workbench provides 2 editors of the data in tabular form:

  • Table editors : Provide single table based access to the data of the query results in the main form
  • Spreadsheets : Provide predefined tabular access including the possibility to query and filter data and can be used as starting form

See a short tutorial for an introduction Video starten with DiversityCollection as an example (Grids are only available in DiversityCollection).

Comparison of tabular forms

Property Table editors Spreadsheets
Range One table Predefined selection of tables
Query Depending on main form Own query
Filter Restricted Direct query in database
Column sequence Fixed Fixed
Start form No Yes
Different versions No Yes
Read only projects No access Read only mode
Data in map No In TK25 sheet

 

Subsections of Data in tables

Spreadsheets

The spreadsheets similar to the grids provide a tabular access to the data. In addition you can search and filter data independent from the main form. For an introduction see two short tutorials Video starten (overview), Video starten (tables and columns). To open a spreadsheet, choose Grid - Spreadsheets - and then one of the provided versions, e.g. Organisms from the menu. A window as shown below will open where the data are marked and organized according to the database tables. If values in one of the tables are missing, this will be indicated with a grey background. The colors correspond to the overview for the database.

Start

To use any of the spreadsheets as a starting window, click on the  button. Now the application will directly open the spreadsheet with the same parameters when you close it. Video starten

 

Database

To change to another database, click on the button and choose the database as described here. Please keep in mind that the main form will connect to the new selected database. 

 

Project

If a user has read only projects available, a button will appear in front of the project. Click on this button to change to the list of projects with read only access. The window will change into the Read only mode (see below). To return to the list of projects with write access, just click no the button again. After changing the source for the project list, the project label will blink with red to remind you, to select a project from the list. Video starten

 

Getting the data

The data are always restricted to one project, selected in the corresponding field (see image above) and restricted to the first top lines as specified in the Max. res. field. To change to the next or previous block of data, use the resp. button. 

The program will organize the data in blocks indicated by the colors as shown above. In the example above the first block contains 2  Organisms all belonging to the block starting with  Hildesheim... for the Event and M-0014196 for the Specimen. The next block then starts with  Hildesheim... indicated by a change in the color containing 2 Organisms. For most of the spreadsheets, the presence of the Event in the data is a prerequisit. As long as you do not sort or filter the data (see below), these blocks will be consistent with the data. As soon as you filter or sort the data, these blocks may be split and may not correspond to the data blocks in the database any more. In the example below you see the effect of sorting by the column Taxon on breaking up the blocks. E.g. the block for the specimen M-0036950 is split into 2.

 

SQL

The query used for the retrieval of the data can be accessed with a click on the SQL button.

 

Filtering

To filter the content of a table you can apply a filter either on the whole table or on each of the columns of the table.

Table filter

This filter will apply on the whole table. Click in the black box next to the filter area and select one of the filter options as shown below.

  • : Filled (all visible columns)
  • : Empty (all visible columns)
  • : Data in table do exist
  • Ø: Data in table do not exist

Tutorial:

Column filter

Tutorial: Video starten

This filter will apply on a single column of a table. To set a filter click in the field underneath the column name. A window as shown below will open. With a click on the button you can enter a value from the current content of the column as filter.

Here you set the sorting (see below), enter the search string and choose among several ways of comparison with the contents in the database:

  • =: The content must be exactly like the given value
  • : The content must be different to the given value
  • ~: The content must be similar to the given value, use wildcards % for any string or _ for a single character
  • ¬: The content must NOT be similar to the given value, use wildcards % for any string or _ for a single character
  • <: The content must be smaller than the given value
  • >: The content must be bigger than the given value
  • |: The content must be in a given list of values
  • : The content must NOT be in a given list of values

For columns linked to modules of the DiversityWorkbench there may be additional options. E.g. for a column linked to DiversityTaxonNames you get 3 additional options:

  • +H: Include lower Hierarchy
  • +S: Include Synonyms
  • +H+S: Include lower Hierarchy and Synonyms

After selecting one of these options, you will be asked for the database and the project where the data should be taken from. In the next step the corresponding names as retrieved from the selected source will be listed (see below). To change the filter click on the button and on the button to remove it.

Sorting

↓↑ Tutorial: Video starten

The data can be sorted by each visible column. If you click into the field underneath the column name a window will open as described above. To change between the modes of sorting, just click on the current sorting. The modes are:

  • -: Not sorted
  • : Sorted in ascending sequence
  • : Sorted in descending sequence

After all parameters are set, the sort mode and filter settings will be shown in the field underneath the column name (see image below). The number indicates the sequence within the sorting columns.

 

Timeout

After all parameters are set, click on the button. If for any reason you get a timeout, meaning the query for the data takes too long, either simplify your query or adapt the time for the query after a click on the button. If you do not want to restrict the time for the queries, set the value to 0. The default value is 30 seconds. 

 

Read only mode

<> Tutorial: Video starten

The formatting of the sheet including the color indication of tables and data blocks is rather time consuming. If these are not needed, e.g. for exporting the data, you can switch to the Read only mode for higher performance. For projects with read only access (see above), the mode is automatically set to Read only

 

Editing

Video starten

To edit the data, just change them. If data are missing, type or select a value in the corresponding field. To remove data, click on the x field. This will remove all entries dependent on this entry that means all data right from the entry in the same line resp. block and any depending data. If there are any depending data, a window will open as shown below listing these data. Click OK if you want to remove the selected data including all depending data as shown in the window.

 

To change the content of several values in a column, either select the whole column with a click on the  button or select the fields manually. To avoid the pop-up of data-entry windows e.g. for columns with a lookup list, click no the button. Once you have selected the contents to change, select the mode of change:

  • Prepend: Insert the given value at the beginning of the content
  • Append: Append the given value at the end of the content
  • Overwrite: Remove current content and insert given value
  • Replace: Replace a string in the content with the given value
  • Clear: Remove the content

After selecting the change mode, enter the text in the field where necessary and click on the corresponding button to perform the changes.

With the context menu (mark area and right click) you can Remove data (for single data fields with a list behind), Tranfer or Copy the content into the clipboard and Insert the content of the clipboard into the selected cells. Tutorial: Video starten

Inclusion of RowGUID

By default the primary key can not be changed in the spreadsheet. In some cases you may want to change parts of the primary key. To do this, you must include the column RowGUID in the list of selected columns. The program will ensure that the changed data do not collide with existing data, that is having the same primary key. See an example in a short tutorial:

 

Fixed sources

Tutorial: Video starten

For columns that are linked to a remote module, you can fix the source for the query. If the source is fixed, the query will directly contact the source as shown in the image below. To set resp. change the source, click on the button. A window will open where you can select either a webservice or a source database together with a project within the database. To remove the source, click on the button (only visible if a source is present).

To see an overview for the fixed sources, click on the button. A window will open where all sources are listed with the source for the current column marked with a yellow background. 

 

A "right-click" on the button in the main form will show the settings as well and with a click on the button you can set the connection for the selected column. If a column depends on values from the data as for example the relations to the module DiversityTaxonNames with a dependence on the taxonomic group, the relation will change with the selected row, depending on the data retrieved from the selected row. If there is no connection to the related module defined, the button will appear gray and if the column has no relation to a module, the button will appear like

If the source is not fixed the query will start as described here.

Settings

- To change the data shown in the query results, either click on the header of the tables, e.g. Event for that table. A window as shown below will open where you can select or deselect columns, change their headers etc. Red columns need to be filled (either by the database, the program or the user). In the Alias field you can rename the columns to your preferences. The button will show a description of the columns. For columns linked to a module, the current settings for the link will be shown in addition to the description. To display a column in the sheet use the checkbox . If a column is needed, but should be hidden, select the checkbox. Displayed columns are indicated by a yellow background while hidden columns get a light yellow background. Certain columns are required (indicated by a purple color) and can not be removed e.g. if they are involved in a link to a remote module like in the example below the columns FamilyCache and OrderCache. Columns linked to a module are indicated by a blue color.

The button allows you to include one of the tables missing in the sheet (see image below). After selection of the table, select the column(s) you want to include in the sheet as shown in the image above.

To store resp. load settings, use the resp. button. If a setting file has been corrupted click on the button to remove it and restart with factory settings. The spreadsheet will always start with the previous settings. The location where the settings are stored are set under Administration - Resources directory ... . For an introduction see a short tutorial Video starten.

 

Column width

Tutorial: Video starten

The width of the columns can be set manually (in the filter area) or automatically with a click on the button. A window will open where you can choose and set the parameters for setting the column width.

 

Adding

Tutorial: Video starten

To add new data, use the area underneath the data table. You can enter preset values that will be used for the new data. If the new data should be added to existing entries, select the corresponding row. Then click on the + field of the datarange that should be inserted. If preset values are given, all corresponding tables will be filled together with the new inserted data. Defaults for the responsibles can be set under Administration - Customize Display. If a column is linked to a remote module, the corresponding columns will be filled together with the selection of the link, so e.g. if you select a link for a taxonomic name to a DiversityTaxonNames the columns for the taxon and if available the familiy etc. will be filled as well. To remove a linked value, select it a choose Remove from the context menu (right click).

Export

Tutorial: Video starten

To export the data as a tab separated text file (UTF8), click on the button. You will be asked if you want to include the hidden columns. These are the primary keys you may need for statistic evaluations etc.

Feedback

To send a feedback to the developer of the software, create a screenshot and click on the  button. 

Details

Tutorial: Video starten

To see all details of a dataset, select the dataset in the sheet and click on the button. A window, corresponding to the main form will open, showing all data related to the selected specimen. 

 

Image

Tutorial: Video starten

If an image is present a preview will be shown in the right upper area of the window. Click on it to see the image in a separate form. The images are restricted to the first image of one source, so e.g. in the organisms sheet only the first image of the specimen linked with this organism will be shown while images of e.g. linked to none or another organism or the collection event will not be addressed. 

Table Editors

For the data selected in the main window the table editors offer a direct access to the tables of the database. The menu Data - Table editors provide an editor for selected tables within the database.

A window with the content of the table will open. Columns with a gray background can not be edited here. Columns with a light gray background are linked to the contents of lookup tables where you can change according to the contents of these tables.

Select Set timeout ... from the menu to increase the default timeout from 5 seconds to a higher value, e.g. for greater amounts of data.  

Editing

You can either edit the contents of the table directly or perform changes to any number of marked fields. To mark a whole column use the  button. Once you have selected the contents to change, select one of the modes of change that appear in the upper left corner. The modes of change are:

  • Insert: Insert the given value at the beginning of the content
  • Append: Append the given value at the end of the content
  • Replace: Replace a string in the content with the given value
  • Clear: Remove the content

After selecting the change mode, enter the text in the field where necessary and click on the corresponding button to perform the changes

Filtering

To filter the content of the table, click in the column that should be used for filtering. Then choose the mode of comparision:

  • = : The content must be exactly like the given value
  • ~ : The content must contain the given value
  • ≠ : The content must be different to the given value

If you want the filtering to be case sensitive, choose the a<>A option. After all parameters are set, click on the button. To undo the filtering, click on the button. This will reset the data to the last saved version. If you want your changes to be saved, click the button before you reset the filtering. If you close the window all changes so far will be saved automatically. So if you do not want to save your changes, click on the button before closing the window.

Export

To export the data as a tab separated text file, click on the button. The file will be automatically saved in the client-folder.  

To export the data in a SQLite database click on the button. The data will be exported into the SQLite database DiversityAgentTables.sqlite in the folder Export in your application directory. If you want to save previous exports, please rename the SQLite database or copy it to a different directory.  

Log data

To see the log data of the table, you can click on the button. The content of the log table can not be changed, but is read only.  

Import/Export

An overview of some options for the im- and export in the module DiversityCollection is shown in the image below. The options Importwizard, Exportwizard, Archive and Replication are available in most of the modules.

Example for a data flow

The image below shows an expample for a data flow from the original source to the final GBIF-portal. As a first step the data are imported via the Import wizard are imported into the database. After the data are given free for publication, they are transferred into the cache database .  From there they are transferred into a Postgres database containing a package for conversion into ABCD. Finally the BioCASE tool for mapping the data is used to provide the data for GBIF.

Subsections of Import/Export

Archive

Creating an archive

The data related to a project can be exported into an archive. Choose Data - Archive - Create archive... from the menu. A window as shown below will open.

Select the project you want to create an archive of and click on the Find the data ="" button. The data related with the project will be imported into temporary tables to allow you to inspect them in advance of the creation of the archive (use the buttons to see the data). To create the archive, click on the Create the archive button. A directory will be created containing a XML file for every table. For a common introduction see the tutorial: Video starten.

You can include the log data by selecting the option as described in the tutorial: Video starten.

 

Resetting the database

Before you restore an archive, please make sure that the data from the archive do not interfere with the data in the database. In order to avoid problems you should clean the database from any user data. To clear the database from any user data, choose  Data - Archive - Reset database... from the menu. A window as shown below will open listing all tables and the number of data within these tables. Click on the Reset database  button to remove any of these data including any data in the log tables.

 

Restoring an archive

To restore an archive choose Data - - Archive - Restore archive... from the menu. A window as shown below will open listing the tables in the database. To restore an archive click on the Choose archive directory button and select the directory containing the archive files. Next click on the Read data ="" ="" button to import the data from the XML files into temporary tables.

With a click on the buttons you can inspect the content of the temporary tables. Finally click on the Restore from archiv button. If you select the option, the import will ask you for a stop in case of an error.

You can include the log data by selecting the option as described in the tutorial: Video starten.  

Planing

Plan schedule based archive creation

To administrate the schedule based creation of archives choose Data - Archive - Administrate archives... from the menu. A window as shown below will open listing the projects in the database. Select the project that should be included in the schedule based creation of archives. To create an archive for all selected projects, click on the Create archives button. The protocol of a previous archiving is shown as in the image below. Successful runs are indicated with a green color while failures have a red background (see below).

 

Creation of xsd schemata

Next to the data, the archive files contain a xsd description of the tables. To create xsd schemata independent of the content, select Data - Archive - Create schema from the menu. A windows as shown below will open with the list of all tables where the main tables of the database are preselected.

To change this selection you may use the  all and  none buttons resp. the Add to selection  and Remove from selection  options using * as a wildcard. Click on the Create schemata   button to create the schemata for the selected tables in the predefined directory. The  button will open this directory containing the created files. The schemata contain the name of the DiversityWorkbench module and its version, the definition of the table, the primary key and the colums together with their datatype and description (see the example below).

Creation of archives as a backgroud process

To archive the data in a scheduler based background process, you can start the application with the following arguments:

  • Archive
  • Server of the SQL-server database
  • Port of SQL-server
  • Database with the source data
  • Optional: Directory where the archive directories should be created

C:\DiversityWorkbench\DiversityAgents> DiversityAgents.exe Archive snsb.diversityworkbench.de 5432 DiversityAgents C:\DiversityWorkbench\DiversityAgents\Archive

The application will create the archives, generate the protocols as described above and quit automatically after the job is done. The user starting the process needs a Windows authentication with access to the SQL-Server database and proper rights to archive the data. If the last argument is not given the default directory …\Application directory\Archive\ will be used.

 

 

 

Replication

Replication

If you wish to work with your data on a local database (called subscriber), e.g. on your laptop, not linked to a database on a central server (called publisher) and these data should be synchronized with the data in the database on the server, you may use the replication function of DiversityCollection. To install the database on your local computer see the installation section.

To use the replication function you require the roles Replicator or Administrator.

Add Publisher

To define a publishing database choose Data -> Replication -> Add Publisher from the menu. A window will open where you choose the publisher. After the publisher is set, you may transfer data between your local database (subscriber) and the publisher. This function is only available for administrators.

Remove Publisher

To remove a publisher from the list choose Data -> Replication -> [Publisher] -> Remove from the menu (where [Publisher] is the name of the publishing database on the publishing server). This function is only available for administrators.

Clean database

Initially you may wish to remove all previous data from your local database (subscriber). Choose Data -> Replication -> Clean database … from the menu. A window will open as shown below where you may choose the ranges which should be cleared:

  • Definitions = the basic definitions within the database, e.g. the available taxonomic groups.
  • Descriptions = the descriptions and their translations of the tables and columns of the database.
  • Project, User = the available projects and users.
  • Basic data = basic data like the collection.
  • Data = the specimen, organisms etc.

Choose the data ranges you wish to clear and click on the button. All tables which contain data will be listed as shown below.

 

Choose the tables which should be cleared and click on the Clean database button. Please keep in mind that you can not delete data from a table as long as there is data in a related table depending on the data you wish to delete. The sequence of the tables is organized to avoid these problems.

Download

To download data from the publisher choose Data -> Replication -> [Publisher] -> Download from the menu (where [Publisher] is the name of the publishing database on the publishing server). A form will open as shown below. Choose the project of the data and the data ranges (see above) which you wish to download. Click on the button to list the tables containing data. To start the download click on the Start download button. With the Force download, ignore conflicts option you can decide whether or not the data in your local database (= Subscriber) should be checked for changes before you download the data from the publisher.

If not all data should be included in the replication, you have the option to set a filter. Click on the button for the table where the data should be filtered to set this filter. A window as shown below will open.

All columns of the table will be listed and allow you to set the filter. To inspect the filtered data, click on the button. Click on the button to see the current filter. If a filter is set this will be indicated with a blue background .  

Merge

To merge data from your local subscriber database with the publisher you must first choose a project. Choose Data -> Replication -> [Publisher] -> Merge** from the menu ([Publisher] is the name of the publishing database on the publishing server). As described for the download, choose the data ranges and click on the button. To start the upload click on the Start merge button.

 

Upload

To transfer data from your local subscriber database to the publisher you must first choose a project. Choose Data -> Replication -> [Publisher] -> Upload from the menu ([Publisher] is the name of the publishing database on the publishing server). As described for the download, choose the data ranges and click on the button. To start the upload click on the Start upload button. With the Force upload, ignore conflicts option you can decide whether or not the data in server (= Publisher) should be checked for changes before you upload the data from your local database (= Subscriber).

As described for the download, data may be filtered with a click on the button (see above).

 

Tools

To fix problems that may interfere with the replication you find some tools under the menu Data -> Replication -> [Publisher] -> Tools… from the menu ([Publisher] is the name of the publishing database on the publishing server). A window will open as shown below.

You may synchronize the RowGUIDs between basic subscriber and publisher tables if for any reason these are differing, e.g. due to manual insert. Choose the table that should be synchronized. The tables will be compared for both publisher and subscriber. The datasets with identical key but different RowGUID will be listed (see above). Click on the Start update button to synchronize the RowGUIDs.  

Conflict

If the transfer of data was successful, the numbers of the transferred data will be shown as below.

During the download or upload a conflict may occur, if the data has been edited in both databases. This will be indicated as shown below.

Click on the button to open a window as shown below where you can choose between the two versions of the data as found in the publisher and the subscriber database.

The conflicting columns are marked red. For text values the program will create a combination of both values (see above) in a merged version of the data. Choose the preferred version of the data and click Solve conflict button. If you can not solve a conflict, use the Ignore conflict or Stop conflict resolution buttons respectively.

 

Report

At the end of each transfer a report will be created with a summary for every table which has been included.

 

Import

There are several import mechanisms available for most modules. Some modules like DiversityDescriptions have special import mechanisms.

Import wizard

for tab separated lists: Import data from foreign sources and attach further data to existing data sets in the database.

Replication

Replication with a local database.

Archive

Restoring an archiv.

Subsections of Import

Import wizard

The examples below are from the module DiversityAgents, but are valid for any other module as well.

With the current solution please ensure that there are no curcurrent imports in the same database.

With this import routine, you can import data from text files (as tab-separated lists) into the database. For short introduction is provided in a video Video starten. Choose Data Import Wizard Agent from the menu. A window as shown below will open that will lead you through the import of the data. The window is separated in 3 areas. On the left side, you see a list of possible data related import steps according to the type of data you choose for the import. On the right side you see the list of currently selected import steps. In the middle part the details of the selected import steps are shown.

 

Choosing the File

As a first step, choose the File from where the data should be imported. The currently supported format is tab-separated text.

Choosing a file will automatically set the default directory for the import files. To avoid setting this directory, deselect the option Adapt default directory in the context menu of the button to open the file.

Then choose the Encoding of the file, e.g. Unicode. The preferred encoding is UTF8.

The Start line and End line will automatically be set according to your data. You may change these to restrict the data lines that should be imported. The not imported parts in the file are indicated as shown below with a gray background. If the First line contains the column definition this line will not be imported as well. To avoid duplicate imports you can Use the default duplicate check for AgentName - see a video Video starten for an explanation.

If your data contains e.g. date information where notations differ between countries (e.g. 31.4.2013 - 4.31.2013), choose the Language / Country to ensure a correct interpretation of your data.

To save all SQL statements that are generated during a test or import, you can check the option Record all SQL statements. Video starten

Finally you can select a prepared Schema (see chapter Schema below) for the import.

 

Choosing the data ranges

In the selection list on the left side of the window (see below) all possible import steps for the data are listed according to the type of data you want to import.

The import of certain tables can be paralleled. To add parallels click on the button (see below). To remove parallels, use the button. Only selected ranges will appear in the list of the steps on the right (see below).

To import information of logging columns like who created and changed the data, click on button in the header line. This will include an additional substeps for every step containing the logging columns (see below). If you do not import these data, they will be automatically filled by default values like the current time and user.

 

Attaching data

You can either import your data as new data or Attach them to data in the database. Select the import step Attachment from the list. All tables that are selected and contain columns at which you can attach data are listed (see below). Either choose the first option Import as new data or one of the columns the attachment columns offered like SeriesCode in the table Series in the example below.

If you select a column for attachment, this column will be marked with a blue background (see below and chapter Table data).

 

Merging data

You can either import your data as new data or Merge them with data in the database. Select the import step Merge from the list. For every table you can choose between Insert, Merge, Update and Attach (see below).

The Insert option will import the data from the file independent of existing data in the database.

The Merge option will compare the data from the file with those in the database according to the Key columns (see below). If no matching data are found in the database, the data from the file will be imported. Otherwise the data will be updated.

The Update option will compare the data from the file with those in the database according to the Key columns. Only matching data found in the database will be updated.

The Attach option will compare the data from the file with those in the database according to the Key columns. The found data will not be changed, but used as a reference data in depending tables. 

Empty content will be ignored e.g. for the Merge or Update option. To remove content you have to enter the value NULL. As long as the column will allow emty values, the content will be removed using the NULL value.

 

Table data

To set the source for the columns in the file, select the step of a table listed underneath the Merge step. All columns available for importing data will be listed in the central part of the window. In the example shown below, the first column is used to attach the new data to data in the database.

A reminder in the header line will show you which actions are still needed to import the data into the table:

  • Please select at least one column   = No column has been selected so far.
  • Please select at least one decisive column   = If data will be imported depends on the content of decisive columns, so at least one must be selected.
  • Please select the position in the file   = The position in the file must be given if the data for a column should be taken from the file.
  • Please select at least one column for comparison   = For all merge types other than insert columns for comparison with data in the database are needed.
  • From file or For all   = For every you have to decide whether the data are taken from the file or a value is entered for all
  • Please select a value from the list   = You have to select a value from the provided list
  • Please enter a value   = You have to enter a value used for all datasets

The handling of the columns in described in the chapter columns.

 

Testing

- To test if all requirements for the import are met use the Testing step. You can use a certain line in the file for your test and then click on the Test data in line:  button. If there are still unmet requirements, these will be listed in a window as shown below.

If finally all requirements are met, the testing function will try to write the data into the database and display any errors that occurred as shown below. All datasets marked with a red background, produced some error.  

To see the list of all errors, double click in the error list window in the header line (see below).

If finally no errors are left, your data are ready for import. The colors in the table nodes in the tree indicate the handling of the datasets:

  • INSERT
  • MERGE
  • UPDATE,
  • No difference
  • Attach
  • No data

The colors of the table columns indicate whether a column is decisive , a key column or an attachment column .  

If you suspect, that the import file contains data already present in the database, you may test this and extract only the missing lines in a new file. Choose the attachment column (see chapter Attaching data) and click on the button Check for already present data. The data already present in the database will be marked red (see below). Click on the button Save missing data as text file to store the data not present in the database in a new file for the import. The import of agents contains the option Use default duplicate check for AgentName that is selected by default. To ensure the employment of this option the column AgentName must be filled according to the generation of the name by the insert trigger of the table Agent (InheritedNamePrefix + ' ' + Inheritedname + ', ' + GivenName  + ' ' + GivenNamePostfix + ', ' + InheritedNamePostfix + ', ' + AgentTitle - for details, see the documentation of the database).

If you happen to get a file with a content as shown below, you may have seleted the wrong encoding or the encoding is incompatible. Please try to save the original file as UTF8 and select this encoding for the import. 

 

Import

- With the last step you can finally start to import the data into the database. If you want to repeat the import with the same settings and data of the same structure, you can save a schema of the current settings (see below). You optionally can include a description of your schema and with the button you can generate a file containing only the description.


Schedule for import of tab-separated text files into DiversityAgents

  • Target within DiversityAgents: Agent
  • Database version: 02.01.13
  • Schedule version: 1
  • Use default duplicate check:
  • Lines: 2 - 7
  • First line contains column definition:
  • Encoding: UTF8
  • Language: US

Lines that could not be imported will be marked with a red background while imported lines are marked green (see below).

If you want to save lines that produce errors during the import in a separate file, use the Save failed lines option. The protocol of the import will contain all settings according to the used schema and an overview containing the number of inserted, updated, unchanged and failed lines (see below).

 

Description

- A description of the schema may be included in the schema itself or with a click on the Import button generated as a separate file. This file will be located in a separate directory Description to avoid confusion with import schemas. An example for a description file is shown below, containing common settings, the treatment of the file columns and interface settings as defined in the schema.


Schedule for import of tab-separated text files into DiversityAgents

  • Target within DiversityAgents: Agent
  • Database version: 02.01.13
  • Schedule version: 1
  • Use default duplicate check:
  • Lines: 2 - 7
  • First line contains column definition:
  • Encoding: UTF8
  • Language: US

Interface settings (= For all)

Table Table alias Column Value Display
AgentContactInformation AgentContactInformation_1 DisplayOrder 1
AgentProject AgentProject_1 ProjectID 0 DiversityWorkbench

 

Subsections of Import wizard

Columns

If the content of a file should be imported into a certain column of a table, mark it with the checkbox.

Decisive columns

The import depends on the data found in the file where certain columns can be selected as decisive. Only those lines will be imported where data are found in [any]{.style1} of these [decisive]{style=“color: #008000”} columns. To mark a column as [decisive]{style=“color: #008000”}, click on the icon at the beginning of the line (see below).

In the example shown below, the file column [Organims 2]{style=“color: #008000”} was marked as decisive. Therefore only the two [lines containing content]{style=“background-color: #99CCFF”} in this column will be imported.

 

Key columns

For the options Merge, Update and Attach the import compares the data from the file with those already present in the database. This comparison is done via key columns. To make a column a key column, click on the icon at the beginning of the line. You can define as many key columns as you need to ensure a valid comparison of the data.

 

Source

The data imported into the database can either be taken From file or the same value that you enter into the window or select from a list can be used For all datasets. If you choose the From file option, a window as shown below will pop up. Just click in the column where the data for the column should be taken from and click OK (see below).

If you choose the For all option, you can either enter text, select a value from a list or use a checkbox for YES or NO.

 

Transformation

The data imported may be transformed e.g. to adapt them to a format demanded by the database. For further details please see the chapter Transformation.

 

Copy

If data in the source file are missing in subsequent lines as shown below,

you can use the Copy line option to fill in missing data as shown below where the [blue values]{style=“color: #00CCFF”} are copied into empty fields during the import. Click on the button to ensure that missing values are filled in from previous lines.

 

 

Prefix and Postfix

In addition to the transformation of the values from the file, you may add a pre- and a postfix. These will be added after the transformation of the text. Double-click in the field to see or edit the content. The pre- and a postfix values will [ only]{.style1} be used, if the [file contains data]{.style1} for the current position.

 
Column selection

If for any reason, a column that should take its content from the imported file misses the position of the file or you want to change the position click on the button. In case a position is present, this button will show the number of the column. A window as shown below will pop up where you can select and change the position in the file.

 

Multi column

The content of a column can be composed from the content of several columns in the file. To add additional file columns, click on the button. A window as shown below will pop up, showing you the column selected so far, where the sequence is indicated in the header line. The [first column]{style=“background-color: #CCFFFF”} is marked with a blue background while the [added columns]{style=“background-color: #8DE089”} are marked with a green background (see below).

To remove an added column, use the button (see below).

 

Information

The button opens a window displaying the information about the column. For certain datatypes additional options are included (see Pre- and Postfix).

Transformation

The data imported may be transformed e.g. to adapt them to a format demanded by the database. A short introduction is provided in a video . Click on the button to open a window as shown below.

Here you can enter 4 types of transformation that should be applied to your data. Cut out parts,  Translate contents from the file, RegEx apply regular expressions or Replace text in the data from the file. All transformations will be applied in the sequence they had been entered. Finally, if a prefix and/or a postfix are defined, these will be added after the transformation. To remove a transformation, select it and click on the button.

 

Cut

With the cut transformation you can restrict the data taken from the file to a part of the text in the file. This is done by splitters and the position after splitting. In the example below, the month of a date should be extracted from the information. To achieve this, the splitter '.' is added and than the position set to 2. You can change the direction of the sequence with the button Seq starting at the first position and starting at the last position. Click on the button Test the transformation to see the result of your transformation.

 

Translate

The translate transformation translates values from the file into values entered by the user. In the example above, the values of the month cut out from the date string should be translated from roman into numeric notation. To do this click on the button to add a translation transformation (see below). To list all different values present in the data, click on the button. A list as shown below will be created. You may as well use the and buttons to add or remove values from the list or the button to clear the list. Then enter the translations as shown below. Use the save button to save entries and the Test the transformation button to see the result. 

To load a predefined list for the transformation use the button. A window as shown below will open. Choose the encoding of the data in you translation source, if the first line contains the column definition and click on the  button to open a file. Click OK to use the values from the file for the translation.

 

Regular expression

The transformation using regular expressions will transform the values according to the entered Regular expression and Replace by vales. For more details please see documentations about regular expressions.

 

Replacement

The replacement transformation replaces any text in the data by a text specified by the user. In the example shown below, the text "." is replaced by "-". 

 

Calculation 

The calculation transformation performs a calculation on numeric value, dependent on an optional condition. In the example below, 2 calculations were applied to convert 2-digit values into 4 digit years.

 

Filter 

The filter transformation compares the values from the file with a value entered by the user. As a result you can either Import content of column in file or Import a fixed value. To select another column that should be compared, click on the button and choose a column from the file in the window that will open. If the column that should be compared is not the column of the transformation, the number of the column will be shown instead of the symbol. To add further filter conditions use the button. For the combination of the conditions you can choose among AND and OR. 

Export

Subsections of Export

Backup

Backup

If you want to create a backup of your database, there are 2 options. You may either export the data as csv files to your local computer or you may create a backup on the server.

Export data as csv

To export your data as csv files to your local computer, choose Data -> Export -> CSV(bcp) ... from the menu. A window will open as shown below, where you can select the tables that should be exported. Click on the Start Export button to export your data. If you choose the option as shown below 2 files will be created for every table. The first file (*.csv) contains the data while the second file (*.xml) contains the structure of the table.

Create backup on the server

To create a backup of your database on the server, choose Data -> Backup database from the menu. This will create a SQL-Server backup on the server where the database is located. Ensure that there is enough space on the server.

Another option is to create a direct copy of the database files on the server. For this you have to use the functions provided by SQL-Server. However, you need administration rights for the database you want to create a backup of. Open the Enterprise Manager for SQL-Server, choose the database and detach it from the server as shown in the image below.

After detaching the database, you can save a copy of the ..._Data.MDF file to keep it as a backup.

After storing the backup you have to reattach the database.

A dialog will appear where you have to select the original database file in your directory.

Export CSV

To export the tables of the database in a tabulator, comma or semicolon separated format, choose Data -> Export -> Export CSV... from the menu. A window as shown below will open where you can select the tables to be exported in sections Selection criteria and in the Tables for export.

A prerequisite for this export is that the bcp program is installed on your computer. This has either been installed together with the installation of SQL-Server or you have to install the Microsoft Command Line Utilities for SQL Server.

To start the export click on the Start export button. By default the data will be exported into a directory ...\Export\<database_name> below your application directory. Click on the button to select a different target directory before starting export.

 

After export the tables are marked with green background, if table schema and data were exported successfully. If only the data were exported, this is marked with yellow background, if nothing was exported, the background is red. A detailed export report can be viewed by a click on the export result file name.  

Export Wizard

Export wizard

The export wizard provides a possibility to export the data selected in the main form. The data are exported as tab separated text file. The export may include transformations of the data as well as information provided by linked modules and webservices. Choose Data - Export - Export wizard from the menu and then select one of the export targets (Event, Specimen, ...). For a short introduction see the tutorial.  

Adding tables

There are the following ways to add tables:

  • One parallel table
  • Several parallel tables according to selected data
  • Dependent table

All options will include the depending tables as defined for the default table. The option for several tables will add as many tables as there are found in the data.

If you added parallel tables, you should set the sequence of the datasets within these tables: For the columns that should be used for sorting the data, set the ordering sequence to a value > 0 and choose if the ordering sequence should be ascending or descending .

 

Certain columns in the database may provide information linked to another table or a module resp. webservice . Click on the button to add a linked value.

 

Adding and editing file columns

To add columns to the exported file, use the buttons. In the textbox at the top of the file column, you can change the header for the column. To change the position of a file column use the resp. button. To fuse a column with the previous column, click in the gray bar on the left side of the column that will change to for fused columns. To remove a file column, use the button. Pre- and postfixes for the columns can directly be entered in the corresponding fields. To apply transformations on the data click on the button.  

Filter

To filter the exported data, use the filter function. Click on the button and enter the text for the filter. Only data matching the filter string will be exported. If a filter is set, the button will have a red background to remind you of the filter. The filter may be set for any number of columns you need for the restriction of the exported data.  

Rowfilter

This filter in contrast to the filter above strictly applies to the row according to the sequence of the data. For an explanation see a short tutorial Video starten.

 

Test

To test the export choose the Test tab, set the number of lines that should be included in the test and click on the Test export button. To inspect the result in a separate window, click on the button.

 

Export

To export your data to a file, choose the Export tab. If you want to store the file in different place use the button to choose the directory and edit the name of the file if necessary. Check the include a schema option if you want to save a schema together with your export. To start the export, click on the Export data   button. To open the exported file, use the button.

 

Export to SQLite

To export your data into a SQLite database, choose the Export to SQLite tab. You may change the preset name of the database in order to keep previous exports. Otherwise you overwrite previous exports with the same filename. To start the export, click on the Export data   button. To view the exported data, use the button.

 

Schema

To handle the settings of your export, choose the Schema tab. To load a predefined schema, click on the button. To reset the settings to the default, click on the button. To save the current schema click on the button. With the button you can inspect the schema in a separate window.

Subsections of Export Wizard

Export Wizard Transformation

Transformation

The exported data may be transformed e.g. to adapt them to a format demanded by the user. Click on the button to open a window as shown below. For an introduction see a short tutorial Video starten.

Here you can enter 6 types of transformation that should be applied to your data. Cut out parts,  Translate contents from the file, RegEx apply regular expressions or Replace text and apply Calculations or Filters on the data from the file. All transformations will be applied in the sequence they had been entered. Finally, if a prefix and/or a postfix are defined, these will be added after the transformation. To remove a transformation, select it and click on the button.

 

Cut

With the cut transformation you can restrict the data taken from the file to a part of the text in the file. This is done by splitters and the position after splitting. In the example below, the month of a date should be extracted from the information. To achieve this, the splitter '.' is added and then the position set to 2. You can change the direction of the sequence with the button Seq starting at the first position and starting at the last position. Click on the button Test the transformation to see the result of your transformation.

With the Start at Pos. option the given splitters will be converted into space (' ') and the whole string starting with the given position will be used (see below).

 

Translate

The translate transformation translates values from the file into values entered by the user. In the example above, the values of the month should be translated from roman into numeric notation. To do this click on the button to add a translation transformation (see below). To list all different values present in the data, click on the button. A list as shown below will be created. You may as well use the and buttons to add or remove values from the list or the button to clear the list. Then enter the translations as shown below. Use the save button to save entries and the Test the transformation button to see the result. 

To load a predefined list for the transformation use the   button. A window as shown below will open. Choose the Encoding of the data in your translation source and indicate if the First line contains column definition. Click OK to use the values from the file for the translation.

 

Regular expression

The transformation using regular expressions will transform the values according to the entered Regular expression and Replace by values. For more details please see documentations about regular expressions.

 

Replacement

The replacement transformation replaces any text in the data by a text specified by the user. In the example shown below, the text "." is replaced by "-". 

 

Calculation 

The calculation transformation performs a calculation on numeric value, dependent on an optional condition. In the example below, 2 calculations were applied to convert 2-digit values into 4 digit years.

 

Filter 

The filter transformation compares the values from the data with a value entered by the user. As a result you can either Export content into file or Export fixed value. To select another column that should be compared, click on the button and choose a column from the file in the window that will open. If the column that should be compared is not the column of the transformation, the number of the column will be shown instead of the symbol. To add further filter conditions use the button. For the combination of the conditions you can choose among AND and OR. 

 

 

 

 

 

 

Export Wizard Tutorial

Export wizard - tutorial

This tutorial demonstrates the export of a small sample from the database. For an introduction see a short tutorial Video starten.

Choosing the data

In the main form, select the data that should be exported (only the data displayed in the query results are exported).

Exporting the data

Choose Data -> Export -> Wizard -> Organism ... from the menu. A window as shown below will open where the available tables for export are listed in the upper left area. To show the data columns of a table, select this table in the list.

 

Adding additional tables

In this example, we want to add as many parallel identification tables as present in the data. To do this, click on the button of the Identification table. At the end of the list (depending on your data) the additional tables are added (see below).

 

Setting the sequence for the tables

To set the sequence of the Identifications, select the first table and for the column IdentificationSequence set sorting sequence to 1 and the direction for sorting to descending

 

Choosing data from linked modules

Some columns provide the possibility to add data from linked tables or modules. In this example we choose the column NameURI linking to the module DiversityTaxonNames (see below).

To provide linked values, click on the button. A window as shown below will open, where you can choose among the provided services.

After the service is selected, you will be asked for the value provided by the service (see below).

Now the selected link is added underneath the column as shown below. You can add as many links as you need for your export.

For some modules there are values that refer to other modules with a name like [Link to ...] as shown in the example below.

If you select one of theses values, you will be asked to select the service or database linked to this modul (see below)

... and then to select one of the provided columns (see below)

Within the form this linked values will be marked as shown below. If several results are retrieved these will be separated with by " | ".

 

Adding columns to the file

To add columns to the exported file, click on the buttons for the columns resp. linked values. In this example select all Family values and the TaxonomicName (see below).

 

Fusing columns

The families should appear as one column and as the sources can exist only once for each identification we can fuse these columns. To do so, click on the delimiters between these columns (see below).

 

Setting the headers

By default the headers for the exported data are set according to the names of the columns in the database. To change this, edit them as shown below where TaxonomicName has been changed to Taxon (see below). For fused columns only the header in the first column will be used.

 

Testing

To test the export, click on the Test export [ ] button. The result depends on the content in your data but should look similar as shown below.

 

Export

To finally export the data, choose the Export tab. By default the data will be exported into tab separated file in a directory in the application directory (see below). You can change the directory (click on the button). You can choose the Include schema option to create a schema that you may reuse in a later export.

Fequently Asked Questions

  • Database
    • Documentation
      • How to adapt the chm manuals for Hugo
        • see chapter …
        • and the video: Anpassungen für HUGO Video starten: Umzug von chm nach html. Anleitung für Umstellung auf html. Übersetzung von html nach markdown. Auswahl der Dateien im Programm. Pandoc als Voraussetzung für Konvertierung. Ansicht der markdown Dateien. Ersetzen des headers (Frontmatter). Anpassungen für Bilder, Ersatz der Icons durch Vektorgrafiken. Beispiel für Vektorgrafik. Bearbeitung der Liste für Ersetzungen über Tabelle. Laden einer Datei mit Ersetungen. Ansicht der Datei und Laden der Datei. Durchfürung der Ersetzungen.
      • How to export the content of enum tables for inclusion in the manual for HUGO
        • see chapter …
        • and the video Ausgabe des Inhalts von Aufzählungstabellen Video starten: Auswahl der Aufzählungstabellen, Anwählen der Ausgabeoption, Auswahl der Spalten, Export und Ansicht im Formular und im Browser.
  • Editing
    • Data in tables
      • Spreadsheet
        • How to include column RowGUID to enable changes in the PK of a table
          • see chapter …
          • and the video RowGUID einschliessen Video starten: Erläuterung der RowGUID. Beispieldaten im TableEditor. Laden der Daten im Spreadsheet. Versuch der Änderung einer Spalte die Teil des Primärschlüssels ist. Einschliessen der RowGUID. Versuch der Änderung mit eingeschlossener RowGUID. Ablehnung der Änderung bei Kollision mit Primärschlüssel. Änderung der Daten ohne Kollision. Ergebnis der Bearbeitung.
  • Import/Export
    • Archive
      • How to create an achive including the log tables
        • see chapter Archiv inclusive des Logs
        • and the video Video starten.
          (Aufbau von Logtabellen, Vorstellung der Beispieldaten, Erstellung eines Archivs: Einschliessen der Logtabellen, Suche nach Daten, Anlegen des Archivs. Wechsel in leere Datenbank. Einlesen der Archivdaten: Auswahl des Archivs, Log einschliessen, Einlesen der Daten, Archivdaten einspielen, Meldung zu vorhandenem Projekt. Anmeldebestätigung. Ansicht der Daten einschliesslich der Daten in den Logtabellen)
      • How to create an archive
        • see chapter Archiv erstellen
        • and the video Video starten.
          (Automatisierte Erstellung durch Server, Manuelle Erstellung, Suche nach Daten, Erstellung, Inhalt einer xml-Datei, Protokoll).

General

Features that are available in several modules of the Diversity Workbench

Subsections of General

Data Access

Access to the data

To get access to the data, you have to fulfill several requirements. In DiversityCollection, you must be a member of one of the user groups. You can only access data, which is listed in the projects you have access to. For external users data may be blocked by entries in the data withholding reasons or due to a data embargo.

   

Subsections of Data Access

Login Administration

Login administration

To administrate the logins on the database server, their permissions and roles respectively as well as access to projects choose Administration - Database - Logins ... from the menu. A window will open as shown below.

To set the website where information about details concerning the General Data Protection Regulation are shown, click on the button.   

To see the current activity on the server click on the button. A window as shown below will open listing all user related processes on the server.

To administrate the  linked servers, click on the button.   

To send a feedback click on the button.   

 

Statistics

To see the activity of a login click on the button. A window will open as shown below listing all databases and tables together with the time span (From - To) and the number of data sets where any activity of the current login has been found.

 

Creation of login

To create a new login click on the button. A window will open as shown below. A login that should be able to create new logins must to be a System administrator.

Here you can enter the name of the new login, the password and the information about the user which will be stored in a DiversityAgents database. You may either create a new entry in this database or select an existing one: Click on the button to search for a name in the database (see below).

 

Copy a login

To copy a login including all permissions etc. into a new login, select the original login in the list and click on the button.

 

Edit a login

To edit the access for a login on the server select the login in the list. If a login should be disabled , uncheck the enabled checkbox (see below).

All databases on the server will be listed with the current database showing a [yellow background]{style=“background-color: #FFFF00”}. The databases where the login has [no access] will be listed in [gray] while the databases accessible for a login are black.

 

Access of a login to a database

To allow the access to a database select the database from the list and choose database as shown below.

The state and date of the privacy consent according to the General Data Protection Regulation is shown in dependence of the selected database.

Roles of a login in a database

Use the > and < buttons to add or remove roles for the login in the database (see below).

To see the detailed permissions of a role, select it in the list of [Available] roles and click on the button. A window as shown below will open listing all objects in the database the role has permissions for (see below).

As a database owner you can edit the permissions and role memberships with the and buttons. Please keep in mind that any change of the permissions may cause serious troubles and should only be used for testing and bug fixing. The final setting of the permissions should be performed by a proper update script of the database. For every action you will get the code that is to be included in an update script (see below).

 

Projects for a login in a database

Depending on the database you can edit the list of projects accessible for a login (see below). Projects are related to the module DiversityProjects. To get additional information about a project select it in the list and click on the button. 

Starting with database version 02.05.35 next to the projects with [full access]{style=“color: #008000; font-weight: bold”}, a second list provides projects with [Read Only] access (see image below). Use the and buttons to move projects between [Accessible]{style=“color: #008000; font-weight: bold”} and [Read Only]{style=“color: #808080; font-weight: bold”}. If a project is set on [Read Only] a user can still add annotations. Starting with version 4.3.219 a project as a whole can be locked with the restriction of access to read only. For more details see chapter Project administration.

To load additional projects click on the Load projects button. A window will open as shown below. Projects already in the database will be listed in [green], [missing projects in red]{style=“color: #FF0000”} (see below). Check all projects you need in your database and click the Start download button.

To see an overview of the users within a project select one of the project in either list and click on the corresponding button . A window as shown below will open listing all users and their roles with access to the selected project.

To add or remove a role for a login, select the corresponding field and choose or from the context menu (see below).

 

Settings of a login in a database

Depending on the database you can edit the settings of a login as shown below.

If you wish to use settings already defined for another login, click on the Search template button. A window (see below) will open where you can choose among the settings defined for logins in the database.

 

Overview for a login

If you want to see an overview of all permissions and project for a login, click on the button. A window as shown below will open. It lists all modules and their databases, the roles, accessible projects and read only projects for a login. 

To copy the permissions and projects of the current login to another login, select the login where the settings should be copied to from the list at the base of the window and click on the button to copy the settings for all databases or the button to copy the settings of the selected database into this login. 

 

Overview for a database

If you see an overview of all user and roles in a database, click on the button. A window a shown below will open. It lists all user, roles and projects in the database. 

To remove a user, select it in the list and click on the button. 

 

Correction of logins

If you select one of the databases, at the base a button may appear. This indicates that there are windows logins listed where the name of the login does not match the logins of the server. This may happen if e.g. a database was moved from one server to another. To correct this, click on the button. A list of deviating logins will be shown, that can be corrected automatically.

If logins with the same name but different server are found, one of them has to be deleted to make the correction possible. You will get a list where you can select those that should be removed.

Select the duplicate logins that should be removed and click OK.

To find users within the database that have no valid login, click on the button. A window as shown below will open, listing the users without a login. Select those that should be removed and click OK. This will include a removal from the collection managers.

 

Security

Security

A user may be in several groups with diverse rights in the database. Here certain higher groups have all rights of lower groups in addition to special rights for the higher group, e.g. the group User may only read data of certain tables while Typist has the rights of User and additionally may edit the data in certain tables - see overview below.

Summarized overview of some of the groups and their permissions as an example for the module DiversityCollection

Role Permissions in addition to lower role and user group respectively Included rights
Administrator Delete data, edit user permissions DataManager
CollectionManager Administration of collections, handling loans etc. StorageManager
DataManager Delete data, edit image descripton templates Editor
Editor Create new entries and delete details (not entire data sets) Typist
Requester Has the right to place requests for specimen
StorageManager Administration of stored parts, handling loans etc. User
Typist Edit data User
User See the data of the data tables, add annotations

To place a user in one of the groups, select Administration - Database - Logins... from the menu. In the window that will open select a login and a database. The roles available in the selected database will be listed as shown below. Use the > and < buttons to add or remove roles for the login in the database (see below).

To see the detailed permissions of a role, select it in the list of [Available] roles and click on the button. A window as shown below will open listing all objects in the database the role has permissions for (see below).

 

If you are an Administrator you may add a user to one of these groups.

Any user may have access to several projects.

Project access for user

The accessibility of projects for users can have 4 different states:

  • No access: The current user has no access to the project
  • Accessible: The current user has access to the project
  • [Read only]: The current user has read only access to the project
  • [Locked]: The project is locked. Any user can either none or read only access to the project

To allow the current user access projects use the [ > ] button for the selected project resp. the [ >> ] button for all projects. To revoke access for the current user use the [ < ]  button for the selected project resp. the [ << ] button for all projects. To change the access for a project to read only use the button and the button to remove a project from the read only list.

 

 

Locking of a project

To lock a selected project use the button. For all users the project will be removed from the accessible or read only list and transferred to the locked list. This is only allowed for a database owner (dbo). Please make sure that you really want to lock a project. Any dataset related to this project will be set to read only for all users. For an introduction, please see the a short tutorial Video starten.

To remove the locked state of a project, select the project in the No access list and click on the  button. The selected project will be moved from the locked list into the read only list for those users that had access to the project.

 

Retrieval of projects from DiversityProjects

Details of the projects within the DiversityWorkbench are stored in the database DiversityProjects. To access further information on a project click on the  button. To edit details in projects you require the application DiversityProjects.exe in your application directory and access to the database DiversityProjects. To synchronize the projects listed in DiversityProjects you may use the synchronize function in the user administration window as shown below. If DiversityProjects is not available, you may create a new project by clicking the button. If DiversityProjects is available, use the synchronize function .

     

License

License

This manual is copyrighted work licensed under a Creative Commons License.

All material in this manual is the property of the contributing authors and fully copyrighted. By choosing this way of publication, the contributing authors have agreed to license the work under a Creative Commons License permitting reproduction, distribution, and derivative works, requiring attribution, notice, and share-alike, and prohibiting commercial use.

 

For information about the license of the client software choose Help, Info…

The client software is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation.

The client software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License (GPL) for more details.

Update

Update of database and client

If either the database or the client needs to be updated, the menu will show an additional entry: Update.

Database update

To update the database, choose Update -> Update database … from the menu. See chapter Database update for details.

Client update

To update the client, choose Update -> Update client … and download the lastest version of the client. ee chapter Update client for details.

Subsections of Update

Update Client

Update of the client software

Replace the files and folders of your installation of DiversityCollection with the files you received by e-mail or downloaded from the DiversityWorkbench portal. The database will not be influenced by this replacement. After starting the new software you need to transfer the settings of the previous version. When you start the program and connect to a database, the program will check if it is compatible with the database or if the database needs an update. In any of these cases an update entry in the menu will appear. If a new version of the client is available, this menu will contain an update client … entry. Click on it to open the webpage where you may download the client as shown below.

 

Update Database

Update database to current version

If you are the owner of the database (Database role = dbo) and the database needs to be updated, the menu will contain an update database … entry. Select this entry to open a window as shown below to run the provided update scripts, delivered with the client software. These scripts need to run consecutively, so e.g. to update from version 2.5.1 to 2.5.4 you either have to run the script DiversityCollectionUpdate_020501_To_020504 or the scripts DiversityCollectionUpdate_020501_To_020502, DiversityCollectionUpdate_020502_To_020503 and DiversityCollectionUpdate_020503_To_020504. The program will guide you through these steps and check for the scripts. All you need to do is click the Start update button. 

Update of all databases on a server

If you are database owner and have a windows login to the database server (user=dbo), you have the option to update all DiversityCollection databases on this server by starting the application with command line parameters. Open a command line window, navigate to the program directory and start DiversityCollection with the keyword “UpdateDatebase”, the server name (or IP address) and the port number:

DiversityCollection.exe UpdateDatabase 127.0.0.1 5432

The program will connect to the server and update all available databases to the current version. If you want to exclude dedicated databases from update, create a text file named “ExcludeDb.txt” that contains the excluded database names - each name in a separate line - and locate it in the resources directory. The update will be done in the background without opening a program window. When the update processing is finished, an overview of the performed actions will be written into the protocol file “Updatereport.log” in directory resources/Updates.

Version

Version

For information about the version of the client application choose Help, Info…

The current version in the example above is 4.0.0 for the client and 2.1.13 for the database

 

 

Administration

Feedback

If you have suggestions for improvement, need any changes in the programm or encouter an error you can give a feedback to the administrator. Click on the ALT and PRINT buttons to get a screen shot of your current form. After creating the screen shot choose Help - Feedback from the menu to open the feedback form as shown below.

 

Click on the Insert image button to insert the screen shot and give a comment about your problem. In the field Topic enter the topic of your feedback (will appear in response e-mail). The Priority can either be empty or set to 3 levels:

  • urgent
  • as soon as possible
  • nice to have

If necessary, enter a date in the field ToDo until when you would like to have the task described in you feedback be solved. Then click on the  Send feedback button to send your feedback to the administrator. If you would like to receive a message when the problem you described is solved, please enter you e-mail address in the field below the description.

To inspect your former feedbacks choose Feedback history from the menu. A window will open where you can browse your past feedback together with the state of progress.

Errorlog

If any error messages show up while working with the application, you can find further details concerning the part of the application where the error occurred and the parameters involved in the file e.g. DiversityCollectionError.log in the Module DiversityCollection located in your resources directory.

To open the errorlog, choose Help - ErrorLog from the menu. A window will open showing the content of the errolog. By default the errorlog will at program start. You can keep the errorlog if needed by chossing Help - Errorlog - Keep error log from the menu. A button will appear that allows you to clear the error log manually: Help - Clear ErrorLog.

Module connections

Connections between the modules of the Diversity Workbench

The DiversityWorkbench is a set of components for building and managing biodiversity information, each of which focuses on a particular domain. DiversityAgents is referred by several modules. To scan for references from these modules on data in DiversityAgents choose Data - Scan modules - from the menu (see image below).

With these options the program will scan all sources of the selected module as listed in the connections for references to the current agent. After selecting an agent in the tree, the sources of the selected modules together with the linked data will be listed as shown below.

Select a link to see a summary of the linked data (see below).

To get further information about an item click on the button. If so far the path to the respective application has not been set, you will get a corresponding message (see below).

Click on the button to set the path to the application (see below).

By default the path to the application is C:\Program Files (x86)\DiversityWorkbench\Diversity...\Diversity.…exe as shown.

Resources

The resources directory is set via the menu (Administration - Resources).

There are 3 possibilities for the resources directory:

  • Select any directory you have read/write access (User defined)
  • Select the "Home" directory of the user
  • Select the "My Documents" directory of the user

The default is set to Home. This directory will contain all files the user need access to (see image below).

Certain directories are hidden (Query) and are handled by the software i.e. the content should not be changed by the user. The other folders are generated by the software if missing, e.g. Export for any exports (see below).

 

Internals

Subsections of Internals

Software versions

The chapters below describe features that are available in several modules of the Diversity Workbench resp. provided via a shared library like DiversityWorkbench.dll

Current version

4.2.11

  • Import wizard
    • Bugfixes for duplicate check for data containing ’ in values of checked column
    • Bugfixes for column pre- and postfix containing ’ in values
    • Option to prevent insert into column
    • Description for columns expanded and redesigned
  • Bugfixes setting linked servers
  • JsonCache: New interface for the modules containing a json summary for datasets

Previous versions

4.2.10

  • Webservice Catalogue of Life available
  • Several bugfixes

4.2.9

  • UserControlQueryList: Dropdownwidth for column selection set to 200 to ensure readablility

WpfControls 1.0.1

  • Bugfix rescanning image if values for height and width are missing
  • Showing message for button ShowDetail if no detail is defined

4.2.8 (2023-08-30)

  • UserControlQueryList - bugfix if for readonly no valid boolean expression is returned
  • Documentation
    • Bugfix - Listing objects
    • Bugfix - Setting title
    • Bugfix - Selecting schema
    • Bugfix init HUGO replacements

4.2.7 (2023-08-17)

  • Formfunctions
    • Bugfix missing command in function public bool initSqlAdapter(ref System.Data.SqlClient.SqlDataAdapter Adapter, string SQL, System.Data.DataTable Table)
    • Bugfix WorkbenchUnit in function public static string getDatabaseNameFromURI(string URI)
    • Bugfix Project.cs - using ViewBaseURL instead of function or linked server
  • Archive
    • Optional inclusion of log tables
    • Bugfixes resetting database
    • Several bugfixes getting data
    • Several bugfixes restoring archive
    • Progress bars included
    • Extension for setting specific query to get IDs for temp table

4.2.6 (2023-08-01)

  • Documentation
    • Bugfix listing objects
    • HUGO
      • Export of enumeration table content
  • Updating depending datasets Video starten.
    • Context menu:
      • Adding current dataset.
      • Adding all datasets from the list.
      • Showing content of the list.
      • Clearing the list.
  • Bugfix setting backlinks for DiversityCollection linked to DiversityAgents
  • Bugfix setting SQL for backlink updates
  • Bugfix setting backlinks for DiversityProjects linked to DiversityAgents
  • Bugfix setting images in tree of FormEnumAdministration
  • FormFunctions: public static string IsoDate(string Text, bool SpaceAsSeparator = true, bool RestrictToDate = false) expanded with optional restriction to date

4.2.5 (2023-07-25)

  • TableEditor:
    • Bugfix in initForm
  • Enumeration administration:
    • Table editor included
  • Agent - Bugfix getting synonyms in unit values
  • Formfunctions - Autocompletion - Bugfix missing Alias for table
  • Documentation
    • HUGO
      • Export as markdown for HUGO included
      • Replacements and adaptions for HUGO included
  • Spreadsheet:
    • Optional inclusion of RowGUID for tables where PK should be edited Video starten
    • Hiding RowGUID by default

4.2.4 (2023-07-12)

  • FormFunctions:
    • public static string IsoDate(string Text, bool SpaceAsSeparator = true)
  • Documentation:
    • Default output set to HUGO
    • HUGO output
      • Index removed
      • Application objects removed
      • ER-diagram located underneath table header
      • Bugfix ER-diagram path
      • Bugfix Enum output
      • Include menu icon

Database

The modules of the Diversity Workbench are based on Microsoft SQL-Server 2014 or above.

Access to database

To access the database select from the menu. A dialog will open, where you can specify the name or IP-adress and port number of the server and the authentication mode.

  

The standard port number for SQL-Server is 1433 and will be set as a default (see left image above). If the database server is configured using a port different from that port, you must give the port number in the field Port (see left image above). If the connection informations are valid, you can choose a database from the server. In the main form you can select the databases that are available under Connection - Database... Video starten

If you access a database for the first time you will be asked to consent to the storage and processing of your personal data (see below) according to the General Data Protection Regulation. Without your consent the access is not possible.

Encryption

The connection to the databases by default is encrypted, symbolized by the icon. To change to an not encrypted connection, click on the icon which will change to accordingly, indicating a not encrypted connection.

 

Subsections of Database

JSON Cache

No-SQL Interface

Databases of the modules provid a No-SQL interface where the data of the main table and the depending tables are merged as JSON.

Table JsonCache

classDiagram
    class JsonCache {
        ID int
        URI varchar 500
        DisplayText nvarchar 500
        LogUpdatedWhen datetime
        Data JSON
    }

This table is filled via the stored procedure procFillJsonCache.

procFillJsonCache is started by a update trigger trgUpd… of the main table in the database

Interface in clients

All modules provide data via a cache table. In the header of the clients you can inspect the content of the JsonCache with a click on the button.

Example

The JsonCache of the current dataset will be shown as in the example below:

[
    {
        "Type": "Taxon",
        "ID": 4403792,
        "Attributes": {
            "FullName": "Arachnida Cuvier, 1812",
            "Rank": "cl.",
            "GenusOrSupragenericName": "Arachnida",
            "Authors": "Cuvier",
            "Publication": ". 1812",
            "Project": [
                {
                    "Project": "SNSBnames"
                }
            ],
            "Checklist": {
                "Analysis": [
                    {
                        "Checklist": "TaxRef_SNSB_NHC-Pests",
                        "Analysis": "IPM | Group",
                        "Value": "70"
                    },
                    {
                        "Checklist": "TaxRef_SNSB_NHC-Beneficials",
                        "Analysis": "IPM | Group",
                        "Value": "70"
                    },
                    {
                        "Checklist": "TaxRef_SNSB_NHC-Bycatchs",
                        "Analysis": "IPM | Group",
                        "Value": "70"
                    }
                ]
            },
            "Synonymy": [
                {
                    "ID": 4403792,
                    "FullName": "Arachnida Cuvier, 1812",
                    "Name": "Arachnida",
                    "Type": "accepted name"
                }
            ],
            "HierarchyListCache": [
                {
                    "HierarchyListCache": "Animalia | Arthropoda | Arachnida"
                }
            ],
            "Hierarchy": [
                {
                    "ID": 4403792,
                    "FullName": "Arachnida Cuvier, 1812",
                    "Name": "Arachnida",
                    "Rank": "cl."
                },
                {
                    "ID": 4403934,
                    "FullName": "Arthropoda Gravenhorst, 1843",
                    "Name": "Arthropoda",
                    "Rank": "phyl./div."
                },
                {
                    "ID": 4403805,
                    "FullName": "Animalia Linnaeus, 1758",
                    "Name": "Animalia",
                    "Rank": "reg."
                }
            ],
            "CommonNames": [
                {
                    "CommonName": "spider",
                    "LanguageCode": "en",
                    "CountryCode": "GB",
                    "Context": "IPM"
                },
                {
                    "CommonName": "Spinnentier",
                    "LanguageCode": "de",
                    "CountryCode": "DE",
                    "Context": "IPM"
                }
            ],
            "Resource": [
                {
                    "URI": "https://museumsschaedlinge.de/gaeste/",
                    "Title": "diverse animals that migrate or are introduced into buildings seasonally, generally don't casue any damage to objects",
                    "Type": "information"
                }
            ]
        }
    }
]

Update

Apart of the update via the Trigger (see below) you can update the JsonCache via the update button underneath the button.

To update the JsonCache for the whole database select Administration - JsonCache… from the menu. a window as shown below will open where you can update the JsonCache for single datasets or the whole database.

Summary

graph TD;
    TaxonName[main table in database] -->|Update in table| trgUpdTaxonName(trgUpd.. of main table in database)
    Mainform[Buttons in main form] -->ButtonUpdateSingle[Update single dataset] --> procFillJsonCache
    Mainform -->ButtonShowSingle[Show single dataset] --> JsonCache
    trgUpdTaxonName --> procFillJsonCache(procFillJsonCache)
    procFillJsonCache -->JsonCache[JsonCache]
    Adminform[Administration form] -->AdminUpdateSingle[Update single dataset] --> procFillJsonCache
    Adminform --> AdminUpdateDB[Update for whole database] --> procFillJsonCache
    Adminform --> JsonCache

OpenAPI

… hier kommt mal swagger hin sobald verfügbar

Configuration

To configure your Database, choose Administration -> Database -> Rename database to change the name of the database according to your requirements. During this renaming all processes in the database will be terminated (you will get a warning if processes from other host are active).

Afterwards you should adapt the address that is published by the database for access by other modules. Choose Administration -> Database -> Set published address from the menu. This will change the published address to the name of the server where your database is located and an identifier for you database, e.g. http://xy.diversityworkbench.de/Collection/

 

Creation

To create or add a database, you need a windows login on the server and sysadmin permissions.

New installation

If you start with a new installation of DiversityAgents connect with the database with a click on the button. Set server and port and as described in the chapter Database access. After connecting to the server, you will get a message, that no database is available and the button will appear. Click on the button to create a new database on the server. A window as shown below will open where you enter the name or the IP-address of the database server. 

127.0.0.1 is the IP-address of your local machine, so in case of a local installation you can keep this value. Next you have to enter the port of the database server. See the chapter Installation of the database for details. 1433 is the default port for SQL-Server. We do recommend to change this port for security reasons.

After server and port are set, the next step is the definition of the database as described below.

 

Adding database

If you are already connected to a database and want to add another database, select Connection - Create database... from the menu. A window will open as shown below where you have to enter the name of the database, starting with DiversityCollection.

After creating the database a window with the script for the creation of the objects in the database (tables, views, functions, ...) as shown below will open.

Click on the button to start the script. After the application of the script you get a message as shown below.

Close the window and you will be connected to the new database. In case there are additional updates available, please start the update of the database to ensure the latest version.

Documentation

Tools for the database documentation

These are the tools to describe the parts of the database and create documentations of the structure. To use these tools, choose Administation - Database - Documentation… from the menu. A window will open as shown below.

Click on the List objects button to list the objects of the database. With the  button resp. button you can select resp. deselect the types in the type selection and the object in the list.

Select the objects that should be listed all button resp. none button you can select resp. deselect the types in the type selection and the object in the list.

Select the objects that should be included in the documentation:

  • Tables
    • Trigger
  • Views
  • Roles
  • Functions and procedures
  • Context

… and exclude the objects that should not be included in the documentation:

  • Logging tables
  • Enumeration tables
  • Old versions of objects
  • System objects
  • Deprecated objects

 

The button Set default seletion will select all items in the list without:

  • System objects
  • Older version of an object indicated by the number at the last position
  • Logging tables
  • Enumeration tables
  • Objects with a description starting with e.g. outdated, deprecated, obsolete etc.
  • HTML options:
    • include index for objects
    • include NULL / NOT NULL
    • include relations and dependencies
    • include Description
    • exclude standard trigger
    • exclude definition
    • include permissions for *_Enum etc.
    • exclude obsolete columns
    • exclude columns starting or ending with the given strings
    • include list of tables that are depending on a table

The buttons Add to seletion and  Remove from seletion  will use the given strings with * as wildcard to add resp. remove items from the selection.

With the Context  option you can show or hide the context area for the html and media wiki tab as shown above.

HTML, MediaWiki, JSP-Wiki

To create a documentation choose among the provided options and click on the button Create ... documentation to create the documentation in one of the available formats (HTML, MediaWiki, JSP-Wiki). 

Subsections of Documentation

chm

With the chm tab you can generate index and keyword html files as well as markdown files for a website generated out of the hhc and hhk files of the HTML Help Workshop for the creation of chm manual as described in the video .

HUGO

In the HUGO / HTML tab you generate markdown files according to HUGO and the relearn theme.

The conversion and adaptions are explained in a short tutorial: Video starten

For enumeration tables the content can be exported as explained in a short tutorial: Video starten

MediaWiki

With the MediaWiki tab you can generate markdown files according to MediaWiki.

Installation

Diversity Workbench modules use Microsoft SQL-Server 2014 or above as database engine. If you do not have a database server with DiversityAgents already available, you have to install the database engine first. Download the free version of Microsoft SQL Server Express 2016 or above from http://www.microsoft.com/downloads/.. Start the program and follow the instructions for the installation.

 

Server configuration

To configure your server for remote access, launch the SQL Server Configuration Manager (see image below).

Then click on the "Protocols for SQLEXPRESS" node. Right click on "TCP/IP" in the list of Protocols and choose "enable" for TCP/IP.

Right click on the TCP/IP node and select, "Properties" to open a window as shown below.

In the part IPALL clear out the value for "TCP Dynamic Ports". Give a TCP-Port number to use when making remote connections, e.g. "4321" as shown above. You have to restart the SQL Server Express service before you can connect to your database. 

If you use a database on a server, make sure that the firewall of the server allows access via the port you set for the connections (see below). 

Start the Microsoft SQL Server Managment Studio and attach the database as shown below. Choose the node "databases" and right-click on it to open the context menu (see below). Then choose "attach" from the context menu. A window will open where you can choose the file DiversityAgents_Data.MDF from your database directory and attach it to the database engine.

 

After the installation make shure to get the latest updates from http://windowsupdate.microsoft.com/.

 

Database configuration

To configure your Database, use the Client as described in

Database configuration.

Linked server

Databases not available on the local server may be accessible via a linked server. Provided the option for loading the connections is set, the program will automatically try to connect to these databases. Otherwise you can connect to these databases as described here. To administrate the linked servers, choose Administration - Linked servers ... from the menu. A form (see below) will open where you can add linked servers and inspect the content of the available databases.

Use the  resp. buttons to add or remove a linked server. To add a linked server, you need the name of the server and the port, e.g. tnt.diversityworkbench.de, 5432, the login associated with the connection of the linked server e.g. TNT and the password for this login. The available databases will be listed as shown above. To inspect the content, select among the tables or views listed in the right part as shown above. Linked servers have certain restrictions for the availability of data, e.g. XML and geography data are not available via a linked server. For a table or view containing incompatible content you may encounter a corresponding error mentioning the reason for the incompatibility.

An expample for a linked server as provided for central databases is shown below (using the management studio for SQL-Server) Linked servers Linked servers

Login administration

To administrate the logins on the database server, their permissions and roles respectively as well as access to projects choose Administration - Database - Logins ... from the menu. A window will open as shown below.

 

Statistics

To see the activity of a login click on the button. A window will open as shown below listing all databases and tables together with the timespan (From - To) and the number of data sets where any activity of the current login has been found.

To see the current activity on the server click on the button. A window as shown below will open listing all user related processes on the server.

 

Creation of login

To create a new login click on the button. A window will open as shown below.

Here you can enter the name of the new login, the password and the information about the user which will be stored in a DiversityAgents database. You may either create a new entry in this database or select an existing one: Click on the button to search for a name in the database (see below).

 

Copy a login

To copy a login including all permissions etc. into a new login, select the original login in the list and click on the button.

 

Edit a login

To edit the access for a login on the server select the login in the list. If a login should be disabled , uncheck the enabled checkbox (see below).

All databases on the server will be listed with the current database showing a yellow background. The databases where the login has no access will be listed in gray while the databases accessible for a login are black.

 

Access of a login to a database

To allow the access to a database select the database from the list and choose database as shown below.

 

Roles of a login in a database

Use the > and < buttons to add or remove roles for the login in the database (see below).

 

Projects for a login in a database

Depending on the database you can edit the list of projects accessible for a login (see below).

There are 4 states of accessibility for projects

  •     Full access: The user can edit the data
  •      Read only access: The user can only read the data
  •      Locked: The project is locked. Nobody can change the data
  •     No access: The user has no access via a project

Projects are related to the module DiversityProjects. To get additional information about a project select it in the the list and click on the button. 

To load additional projects click on the Load projects button. A window will open as shown below. Projects already in the database will be listed in [green]{style=“color: #008000”}, [missing projects in red]{style=“color: #FF0000”} (see below). Check all projects you need in your database and click the Start download button.

 

 

Overview for a login

If you see an overview of all permissions and project for a login, click on the button. A window a shown below will open. It lists all modules and their databases, the roles, accessible projects and read only projects for a login. 

To copy the permissions and projects of the current login to another login, select the login where the settings should be copied to from the list at the base of the window and click on the button to copy the settings for all databases or the button to copy the settings of the selected database into this login. 

 

Overview for a database

If you see an overview of all user and roles in a database, click on the button. A window a shown below will open. It lists all user and roles in the database. 

To remove a user, select it in the list and click on the button. 

 

 

Correction of logins

If you select one of the databases, at the base a button may appear. This indicates, that there are windows logins listed where the name of the login does not match the logins of the server. This may happen if e.g. a database was moved from one server to another. To correct this, click on the button. A list of deviating logins will be shown, that can be corrected automatically.

If logins with the same name but different server are found, one of them has to be deleted to make the correction possible. You will get a list where you can select those that should be removed.

Select the duplicate logins that should be removed and click OK.

Tools

These are the tools to handle the basic parts of the database. These tools are only available for the owner of the database and should be [handled with care]{.style2} as any changes in the database may disable the connection of your client to the database. Before changing any parts of the database it is recommended to backup the current state of the database. To use these tools, choose Administation -> Database -> Database tools ... from the menu. A window will open as shown below.

Description

The Description section (see above) shows the basic definitions of the objects in the database and enables you to enter a description for these objects including tables and their columns, functions and their parameter etc. With the buttons SQL for adding, update and both you can generate SQL statements for the creation of the descriptions in your database. Use the button both if you are not sure if a description is already present as it will generate a SQL statement working with existing and missing descriptions (see below).

The button  Fill Cache  fills the table CacheDescription where all descriptions are collected for easy access.

 

Log table and trigger

In the Log table and trigger section (see below) click on the List tables button to see all tables within the database. The Table section shows the basic definitions of a selected table. If columns for logging the date and responsible user for inserting and updating the data are missing, you can use the Attach ... button to attach these columns to the table. Furthermore you may add a RowGUID to the table as e.g. a preparation for a replication.

In the Log table section (see below) you can create a logging table for the selected table in a format as used within the Diversity Workbench. Click on the Show SQL ... button to show the SQL-statement that will create the logging table. If an old logging table should be kept, choose the Keep old log table option. If your table should support the version setting from a main table, choose the Add the column LogVersion option. To finally create the logging table click on the Create LogTable ... button.

The triggers for insert, update and delete are created in the according sections (see below). If an old trigger exists, its definition will be shown in the upper part of the window. Click on the Show SQL button to see the definition of the trigger according to the current definition of the table in a format as used in the Diversity Workbench. If a trigger should set the version in a main table, which the current table is related to, choose the Add version setting to trigger option. To enable this option you must select the version table first. To finally create the trigger click on the Create trigger button. The update and delete triggers will transfer the original version of the data into the logging tables as defined above, where you can inspect the history of the data sets.

If so far no procedure for setting the version in a main table is defined, you can create this procedure in the last section. Click on the Show SQL button to see the definition and on the Create Procedure button to create the procedure (see below).

 

Preparation for replication

If you want to use replication within you module, the tables need certain columns and a log table. These preparations can be performed by a script, generated in the section Replication (see below). Select the tables you want to include in the process and create the script. This script can than be included in an update of the database. Please ensure that these changes are only be done by expert staff.

 

Clear logtables

If for any reason you want to clear the log tables of the database, this can be done in the Clear log tab as shown below. Click on the List tables button to list the log tables. Then select those that should be cleared and click on the Clear log of selected tables button (see below). Please keep in mind that any restoration of data from the log is only possible as long as the data can be retrieved from the log.

 

Data protection

To implement the General Data Protection Regulation of the European Union several steps have to be performed in a database:

  • Generate a skript using this tool (see below) to convert all tables and insert objects according to the requirements:
    • Add columns ID, PrivacyConsent and PrivacyConsentDate in table UserProxy
    • Grant update to PrivacyConsent and PrivacyConsentDate in table UserProxy
    • Create update trigger for UserProxy setting the PrivacyConsentDate
    • Create the function providing the ID of the user from UserProxy
    • Create the function providing the name of the user from UserProxy
    • Create the function PrivacyConsentInfo providing common information
    • For every table:
      • Insert users from a table into UserProxy (if missing so far)
      • Insert users from the corresponding log table into UserProxy (if missing so far)
      • Change the constraints for the logging columns (User_Name() → UserID())
      • Replace user name with ID in logging columns
      • Replace user name with ID in logging columns of the log table
      • Adapt description of the logging columns
  • Include the skript in an update of the database
  • Check the database for update triggers, functions using e.g. CURRENT_USER, USER_NAME, SUSER_SNAME etc. where user names must be replaced with their IDs. Create a script performing these tasks and include it into an update for the database
  • Adapt the client to the now changed informations (e.g. query for responsible etc.)

After these changes the only place where the name of a user is stored is the table UserProxy together with the ID. Removing the name (see below) will remove any information about the user leaving only a number linked to the information within depending data.

To generate a script for the objects and changes needed to implement the General Data Protection Regulation use the Data protection tab as shown below. The generated script will handle the standard objects (logging columns) but not any additional circumstances within the database. For these you need to inspect the database in detail and create a script to handle them on your own.

To set the website where detailed information about the handling of the General Data Protection Regulation within the DiversityWorkbench resp. the current database is provided, click on the button on the Info site tab. If unchanged, the default site for the DiversityWorkbench is set (see below).

If for any reason a user wants his name to be removed from the database, select the users name from the list as shown below and click on the  button (see below).

History

To inspect the history of a dataset click on the button. A form will open, showing all former states of the data in the tables with the current dataset at the top. The version is shown in the header of the main.

The version will be set automatically. If a dataset is changed the version will be increased if the last changes where done by a different user or the last change is more than 24 hours ago (for further details see topic Logging ).

For analysis of the succession of changes the log tables contain additional columns:

  • Kind of change: This column is set by the trigger inserting data into the log table
    • current version: This is the current state of the data in the table
    • UPDATE: This is the state of the data before an update happened
    • DELETE: This is the state of the data when the data have been deleted
  • Date of change: The date and time of the changes. This column has the default value getdate() that means the current date an time is set when any data are inserted into the log table
  • Responsible: The user reponsible for the changes. This column has the default value suser_sname() that means the current user is set when any data are inserted into the log table
  • LogID: A unique ID of the logtable. This column is an identity that means it is set by the database when any data are inserted into the log table

Logging

Changes within the database will be documented for each dataset with the time and the responsible user in the columns shown in the image below.

All main tables have a corresponding logging table. If you change or delete a dataset the orignial dataset will be stored in this logging table together with informations about who has done the changes and when it happend. To see the data stored in the logging tables, click on the button to open the history of a dataset.

Webservices

Webservice - foreign sources

Some modules within the DiversityWorkbench provide the possibility to link your data to an external webservice. For example DiversityTaxonNames gives you access to the taxonomic names of Index Fungorum. To establish a connection to an external webservice click on the button. A window will open where you may choose from either DiversityWorkbench modules or external Webservices. For an overview see a short tutorial . The currently provided webservices are:

Subsections of Webservices

Catalogue Of Life

DiversityWorkbench provides the possibility to link your data to an external webservice. The webservice provided by the Catalogue of Life is possible through the module DiversityTaxonNames. To establish a connection to this external webservice click on the button. A window will open where you may choose this webservice (see below).

In the field Name in Query conditions enter your search string and click on the button to start the query. In the list of the left upper part the results of the query will be listed. In the right part of the window additional information is shown as provided by the webservice. To include the link from the webservice into your database choose one of the entries in the list and click OK. The entry will change as shown below.

If you double-click on the link area, a window will open providing you with the retrieval information of the webservice (e.g.:https://api.checklistbank.org/dataset/3LR/nameusage/8K9Y). 

To receive the information related to an entry as provided by the webservice click on the button. A window will open as shown below where information on the webservice is listed in the upper part. If available, additional information provided on a corresponding website will be shown in the lower part.

If you wish to remove the link to the webservice, click on the button. This will only remove the relation to the webservice, not the cached name.

 

 

 

 

 

Subsections of Catalogue Of Life

Webservice Palaeo DB

The Palaeontology Database - webservice

DiversityWorkbench provides the possibility to link your data to an external webservice. The webservice provided by the Palaeontology Database is possible through the module DiversityTaxonNames. To establish a connection to this external webservice click on the button. A window will open where you may choose this webservice (see below).

In the field Name in Query conditions enter you search string and click on the button to start the query. In the list of the left upper part the results of the query will be listed. In the right part of the window additional information is shown as provided by the webservice. The lower part of the window will show the webpage of the related information.

For synonyms the accepted name will be shown as well as shown above. To change to the accepted name click on the linked entry of the ID - in the example above ID: 57254. To include the link from the webservice into your database choose one of the entries in the list and click OK. The entry will change as shown below.

If you double-click on the link area , a window will open providing you with the retrieval information of the webservice. 

To receive the information related to an entry as provided by the webservice click on the button. A window will open as shown below where information on the webservice is listed in the upper part. If available, additional information provided on a corresponding website will be shown in the lower part.

If you wish to remove the link to the webservice, click on the button. This will only remove the relation to the webservice, not the cached name.

 

 

 

 

 

Webservice Index Fungorum

Some modules within the DiversityWorkbench provide the possibility to link your data to an external webservice. For example DiversityTaxonNames gives you access to the taxonomic names of Index Fungorum. To establish a connection to this webservice, click on the button. A window will open where you may choose Index Fungorum from the database list (see below). 

Enter the query restriction for the name in the Name field in Query conditions. The maximum number of records you receive may be set in the max. results field (choose a low number, if you have a slow connection to the internet). Click on the search button to start the query. In the list in the left upper part the results of the query will be listed. In the right part of the window additional information is shown as provided by the webservice. For certain entries buttons will appear, as e.g. shown above for the basionym and the current name of a scientific name. Click on these buttons, if you wish to change to one of these related data sets from the webservice. If available, the information provided on the corresponding website is shown in the lower part. To include the link from the webservice into your database choose one of the entries and click OK. The entry will change as shown below.

If you double-click on the link area , a window will open providing you with the retrieval information of the webservice. 

To receive the entire information related to this entry as provided by the webservice click on the button. A window will open as shown below where information on the webservice is listed. If available, the lower part will show the corresponding information of a website.

If you wish to remove the link to the webservice, click on the button. This will only remove the relation to the webservice, not the cached name.

Webservice PESI

Pan-European Species directories Infrastructure

Some modules within the DiversityWorkbench provide the possibility to link your data to an external webservice. For example DiversityTaxonNames gives you access to the taxonomic names of the Pan-European Species directories Infrastructure (PESI). To establish a connection to this webservice, click on the button. A window will open where you can choose PESI from the database list (see below). 

Enter the query restriction for the name in the Name field in Query conditions. Click on the search button to start the query. In the list in the left upper part the results of the query will be listed. On the right part of the window additional information is shown as provided by the webservice. If available, the information provided on the corresponding website is shown in the lower part. To include the link from the webservice into your database choose one of the entries and click OK. The entry will change as shown below.

If you double-click on the link area , a window will open providing you with the retrieval information of the webservice. 

To receive the entire information related to this entry as provided by the webservice click on the button. A window will open as shown below where information on the webservice is listed. If available, the lower part will show the corresponding information of a website.

If you wish to remove the link to the webservice, click on the button. This will only remove the relation to the webservice, not the cached name.

Webservice RLL

Recent Literature on Lichens - webservice

The module DiversityReferenes gives you access to the webservice Recent Literature on Lichens. To access this service click on the button. A window will open where you can search for entries in the database (see below). 

Enter the query restriction for the Authors, the Year and the Title in the query conditions. Click on the search button to start the query. In the list in the left upper part the results of the query will be listed. On the right part of the window additional information is shown as provided by the webservice. To include the link from the webservice into your database choose one of the entries and click OK. The entry will change as shown below.

If you double-click on the link area , a window will open providing you with the retrieval information of the webservice. 

To receive the entire information related to this entry as provided by the webservice click on the button. A window will open as shown below where information on the webservice is listed.

If you wish to remove the link to the webservice, click on the button. This will only remove the relation to the webservice, not the cached name.

Adaptions for HUGO with theme relearn

Installation

Update des Themes

um das Theme auf die letzte Version zu bringen kann man den Befehl git submodule update --remote --merge themes/relearn verwenden

Übersetzung des Bestands an html

  • Übersetzung der *.html Seiten mit pandoc in *.md
  • Aufbau einer Ordnerstruktur die dem Index der chm Datei entspricht
  • Das Basisdokument der Ordner wird in die Ordner verschoben und in _index.md umbenannt
    • Dort im Frontmatter steht der Titel der im Menü angezeigt wird, e.g.:
      --- 
      title: Installation 
      ---  

Überarbeitung der md Dateien

  • Korrektur der Bildverweise
    • Ordner mit den Bildern in den Ordner static kopieren
    • von e.g. ![](img/...) in ![](/manual/dwb/img/...)
    • ACHTUNG - Case sensitiv. Namen müssen stimmen
    • Icons gegebenenfalls freistellen für Darkmode
  • Entfernung aller störenden Formatierungsangaben
  • Entfernung der Kopfzeile (Überschrift wird von HUGO automatisch erzeugt)
  • Korrektur der internen Verweise
    • ändern von [![](/manual/dwb/img/VideoDE.svg?classes=inline)](http://media.snsb.info/Tutorials/dwb/Editing/OeffentlicheKontaktdaten.webm) zu [![Video starten](/manual/dwb/img/VideoDE.svg?height=20px&lightbox=false&classes=inline)](http://media.snsb.info/Tutorials/dwb/Editing/OeffentlicheKontaktdaten.webm)
      • ansonsten wird das Bild gezeigt statt das Video zu starten
    • ändern von
      [Contact](Contact.htm)
      zu e.g.
      [Contact](/manual/dwb/editingdata/contact)
    • Wenn als Basisadresse in hugo.toml etwas angegeben wurde, e.g. baseURL = "http://www.diversityworkbench.de/manual/dwb/" dann muss diese auch für Verweise innerhalb der Files verwendet werden.
      • e.g. Bildverweise ![](/manual/dwb/img/IcoFeedback.gif?classes=inline)
      • Dateiverweise [Anmelden](/manual/dwb/database)
      • HUGO relearn erzeugt für Überschriften Anker die man ansteuern kann, e.g. kann man ### Table **AgentResource** über die Adresse /manual/dwb/database/database/#table-agentresource erreichen. Ein Index Eintrag dafür wäre e.g. [AgentResource](/manual/dwb/database/database/#table-agentresource). ACHTUNG - Case sensitiv: ### Table **AgentResource** wird in #table-agentresource übersetzt
    • Kommentare starten mit # ohne folgendes Leerzeichen

Frontmatter

You can change the frontmatter to a default using the documentation tool

  • Steht am Anfang der Datei und ist bei yaml durch --- oben und unten abgegrenzt, e.g.
    ---
    title: Login administration
    linktitle: Logins
    weight: 5
    menuPre: <img src="/manual/dwb/img/Documentation.svg" height="20">&nbsp;
    alwaysopen: false
    ---
  • Seiten die noch in Entwicklung sind kann man mit draft: true im Frontmatter markieren. Diese werden dann nicht in die Ausgabe übernommen
  • Der Titel wird mit title: Login administration angegeben. Dieser erscheint dann auch in der Seite als Überschrift
  • Der Text im Menü kann abweichend definiert werden mit linktitle: Logins. Ansonsten erscheit der Titel im Menü
  • Die Reihenfolge im Menü kann mit weight: 5 angegeben werden. Ansonsten wird alphabetisch sortiert
  • Ein Logo kann man mit menuPre: <img src="/manual/dwb/img/LinkedServer.svg" height="20">&nbsp; hinzufügen. Das Bild sollte *.svg sein
  • Wenn das Untermenue erst beim Anwählen geöffnet werden soll: alwaysopen: false

Bilder

You can adapt the images to a default using the documentation tool

  • Icons die e.g. in den Text integriert werden sollen, müssen folgedermassen eingebaut werden:
    • ![](/manual/dwb/img/Database.svg?height=20px&lightbox=false&classes=inline)
  • Die Bilder am Anfang der Seite werde wie folgt eingebaut:
    • ![](/manual/dwb/img/LinkedServer.svg?height=10vw&lightbox=false)

mit px wird das Bild mitgezoomt, bei vw bleibt es gleich gross

  • noch nicht zu svg konvertierte Bilder die im Fliesstest erscheinen sollen werden wie folgt eingebunden:
    • ![](/manual/dwb/diversitycollection/IcoDelete.gif?classes=inline&lightbox=false)
  • sonstige Bilder mit
    • ![](/manual/dwb/diversitycollection/IcoDelete.gif)

mit der Angabe ...lightbox=false wird verhindert, dass ein Bild beim Anklicken mit der Maus geöffnet wird. Dies sollte bei Bildern die nicht nach svg konvertiert wurden und nicht im Fliesstext erscheinen nicht verwendet werden, damit der User bei kleinen Bildern diese in Originalauflösung betrachten kann. Unten 2 Beispiele

![](/manual/dwb/diversitycollection/IcoDelete.gif?classes=inline&lightbox=false)

![](/manual/dwb/diversitycollection/IcoDelete.gif?classes=inline)

Für Links innerhalb des Manuals kann man shortcodes verwenden. Dafür entweder auf den Namen der Datei oder auf Links von Überschriften (ab ##) verwenden. Diese müssen innerhalb des Manuals eindeutig sein. Für Header als erstes Zeichen # dann Überschrift und alles lower case und Leerzeichen werden durch - ersetzt. Beispiel:

## Main form of diversityexsiccatae

wird zu sofern es sich in der gleichen Datei befindet: {{ % relref "#main-form-of-diversityexsiccatae" % }}

Für Links ausserhalb der Datei werden Verweise unter Einschluss des Dateinamens verwendet:

Verweis auf ein Kapitel innerhalb einer Datei {{ < relref "diversityexsiccatae#main-form-of-diversityexsiccatae" > }}

bzw. nur auf die Datei {{ < relref "diversityexsiccatae" > }}

Leerzeichen zwischen {{ % und % }} entfernen

Von ausserhalb kann e.g. eine Überschrift mit https://www.diversityworkbench.de/manual/dwb/modules/diversityexsiccatae/index.html#main-form-of-diversityexsiccatae aufgerufen werden. Diese können direkt aus dem Manual kopiert werden.

  • hierfür das Logo in den Ordner static kopieren
  • im Ordner layouts einen Ordner partials anlegen
  • dort eine Datei logo.html anlegen
    • in dieser auf das Logo verweisen e.g.:
      <h4><b>DiversityAgents</b></h4>
      <img src="/DA_4D.svg">
  • in static - layouts - partials die Datei menu-footer.html anlegen und anpassen

favicon

Im Ordner static den Ordner images anlegen Datei favicon.ico in der Ordner static/images kopieren

ER-Diagramm

dieses kann als Mermaid eingebaut werden, e.g.

---
title: ER-Diagram
---
graph LR;
    A[Agent] --> B[AgentContact<br/>Kontaktdaten der Agents]
    A --> C[AgentReference]
    A --> D[AgentIdentifier]
    A --> E[AgentResource]
    A --> F[AgentExternalID]
    G[AgentExternalDatabase] --> F[AgentExternalID]

soll das Diagramm zoombar sein wird die Version 5.23 des Themes benoetigt. Ausserdem kann der Parameter nur für die Shortcode Version angegeben werden, nicht für die Codefences:

{{ < mermaid align="center" zoom="true" > }}
... 
(remove space between {{ and < resp > and }} in header and footer for correct code)
...
{{ < /mermaid > }}
---
title: ER-Diagram
---
graph LR;
    A[Agent] --> B[AgentContact<br/>Kontaktdaten der Agents]
    A --> C[AgentReference]
    A --> D[AgentIdentifier]
    A --> E[AgentResource]
    A --> F[AgentExternalID]
    G[AgentExternalDatabase] --> F[AgentExternalID]

Anpassung des Themes

  • es werden 2 eigene Themes bereitgestellt

    • im Verzeichnes
      • themes
        • relearn
          • static
            • css:
            • theme-dwb-dark.css
            • theme-dwb.css

    diese an DWB Anforderungen anpassen

    • in \themes\relearn\static\css\theme.css
      #body img.inline {
          display: inline !important;
          margin: 0 !important;
          vertical-align: middle;
          /* vertical-align: bottom; */
      }
    • in \themes\relearn\static\css\theme-dwb.css
      /*--MENU-HEADER-BG-color: rgba( 28, 144, 243, 1 );*/ /* Background color of menu header */
      --MENU-HEADER-BG-color: rgba( 220, 220, 220, 1 ); /* Background color of menu header */
      --MENU-HEADER-BORDER-color: rgba( 51, 161, 255, 1 ); /*Color of menu header border */
      
      --MENU-SEARCH-color: rgba( 255, 255, 255, 1 ); /* Color of search field text */
      /*--MENU-SEARCH-BG-color: rgba( 22, 122, 208, 1 );*/ /* Search field background color (by default borders + icons) */
      --MENU-SEARCH-BG-color: rgba( 90, 90, 90, 1 ); /* Search field background color (by default borders + icons) */
      /*--MENU-SEARCH-BORDER-color: rgba( 51, 161, 255, 1 );*/ /* Override search field border color */
      --MENU-SEARCH-BORDER-color: rgba( 0, 0, 0, 1 ); /* Override search field border color */

Konfiguration - in hugo.toml:

```native
baseURL = "http://www.diversityworkbench.de/manual/dwb/"
languageCode = "en-us"
title = "DiversityAgents"
theme = "relearn"

[outputs]
    home = ["HTML", "RSS", "SEARCH", "SEARCHPAGE"]
    section = ["HTML", "RSS", "PRINT"]
    page = ["HTML", "RSS", "PRINT"]

[params]
    themeVariant = [ "auto", "dwb", "dwb-dark" ]
```

Start des Testservers:

  • mit einem Terminal in das Verzeichnis des Projekts wechseln
  • dort hugo server eingeben.
    • bei Problem mit Sonderzeichen: den Inhalt der Datei config.toml in hugo.toml kopieren und config.toml löschen (beide sollten wenn vorhanden UTF8 sein - werden manchmal als UTF16 angelegt - dieses dann nach UTF8 ändern)
      • Error: “…\diversityworkbench\hugo.toml:1:1”: unmarshal failed: toml: invalid character at start of key: ÿ
  • Im Browser an die angegebene Adresse navigieren, e.g. localhost:1313
  • Wenn als Basisadresse in hugo.toml etwas angegeben wurde, e.g. baseURL = "http://www.diversityworkbench.de/manual/dwb/" dann muss die passende Adresse eingeben werden also e.g. localhost:1313/manual/dwb/