v4.3

Overview:

This release added one major feature, DataCite API support and several bug fixes and feature enhancements:

• Added support for creating DOIs using the DataCite API.
• Improved the installer to include more configuration steps.
• Addressed a potential security issue involving the default configuration.
• #2115 Allow log in using email address in addition to username.
• Check for email address uniqueness regardless of case.
• Improved performance for the zip download of a large number of files.
• Allow customizable text for publish dataset popup.
• Improved performance of user name look up on permissions pages.
• Allow hiding explore button when two ravens is not configured.
• Allow disabling google analytics if not needed.
• Support using DOI as identifier in dataset APIs.
• Fixed a URL targeting issue for citation links inside a dataverse widget and iframe on remote sites.

Please see the <a href=https://github.com/IQSS/dataverse/issues?q=milestone%3A4.3+is%3Aclosed>4.3 milestone issues list for more information.

Installation:

If this is a new installation, please see our Installation Guide.

Upgrade:

If you are upgrading from v4.x, you must upgrade to each intermediate version before installing this version. When upgrading from the previous version, v4.2.4 you will need to do the following:

1. First, check for and correct duplicate email addresses, including duplicates due only to different case. This is a new requirement for this version.
   • See remove_duplicate_emails.txt for more information.
2. Undeploy the previous version.
   • /glassfish4/bin/asadmin list-applications
   • /glassfish4/bin/asadmin undeploy dataverse.war
3. Deploy this version.
   • /glassfish4/bin/asadmin deploy /dataverse-4.3.war
4. Run the database update script.
   • psql -U <db user> -d <db name> -f upgrade_v4.2.4_to_4.3.sql
5. Restart glassfish.
   • service glassfish stop
   • service glassfish start
6. If you were running Two Ravens, enable it for this version.
   • From the glassfish server console, run:
     curl -X PUT -d true http://localhost:8080/api/admin/settings/:TwoRavensTabularView
7. Run the incremental index to correct the targeting for citation links.
   • Remove last index times from the database:
     update dvobject set indextime=null;
     update dvobject set permissionindextime=null;
   • From the glassfish console, run the incremental index:
     curl -X GET "http://localhost:8080/api/admin/index/continue?numPartitions=1&partitionIdToProcess=0"

If you are upgrading from v3.x, you will need to perform a migration to v4.x since our application was redesigned and the database schema are completely different. This is a significant undertaking. Please contact us (support at dataverse.org) before beginning. Also refer to our migration google group for additional support and information: https://groups.google.com/d/msgid/dataverse-migration-wg

Please note: v4.x does not currently support creating new handles though it will support existing ones. We intend to add this feature but have not yet scheduled this work.

v4.2.4

Overview:

This is a patch release to address issues with harvested dataset links and correct the behavior of zip downloads when selecting restricted files to which you do not have permission. There are a few other corrections such as fixing the :ZipDownloadLimit setting, allowing a Dataverse installation administrator to adjust the maximum amount of data that can be downloaded at one time. We have also reorganized our Installation Guide and added significantly to the content.

For the complete list of issues, see the <a href=https://github.com/IQSS/dataverse/issues?q=milestone%3A4.2.4+is%3Aclosed>individual issues.

Installation:
Please see our Installation Guide

Upgrades:
If upgrading from v3.6.2 or earlier, please contact us at [email protected] since this requires considerable planning and effort.

If upgrading from v4.x, you will need to follow the upgrade instructions for each intermediate version.
This version requires you to first undeploy the current application, then deploy this war file.

1. undeploy the current application
   /glassfish4/bin/asadmin list-applications
   /glassfish4/bin/asamin undeploy dataverse-4.2.3
2. deploy the new war file
   /glassfish4/bin/asadmin deploy dataverse-4.2.4.war

v4.2.3

This release is small in scope; it's mostly about a technology upgrade that includes moving to Java 8 and upgrading to Prime Faces 5.3. We have fixed a number of issues, including making the file ingest page refresh more reliable.

For more information see the <a href=https://github.com/IQSS/dataverse/milestones?state=closed>project milestone notes and <a href=https://github.com/IQSS/dataverse/issues?q=is%3Aissue+milestone%3A4.2.3+is%3Aclosed>individual issues.

IMPORTANT, for both new installations and upgrades:

For the file uploads to work properly, please follow the instructions in the README.txt file in https://github.com/IQSS/dataverse/tree/master/conf/jhove (In future version of the Dataverse the installer script will be taking care of this).

Installation:
Please see our Installation Guide

Upgrades:
If upgrading from v3.6.2 or earlier, please contact us at [email protected] since this requires considerable planning and effort.

If upgrading from v4.x, you will need to follow the upgrade instructions for each intermediate version.
This version requires you to install Java 8, undeploy the current application, then deploy this war file.

1. undeploy the current application
   /glassfish4/bin/asadmin list-applications
   /glassfish4/bin/asamin undeploy dataverse-4.2.2
2. deploy the new war file
   /glassfish4/bin/asadmin deploy dataverse-4.2.3.war

Please note that this version of Dataverse requires Java 8 to be installed, otherwise the application will not deploy. See the java section of the Installation Guide.

If you are using shape files and GeoConnect functionality, please run the following commands from the web server console since it has been made disabled by default:

Set GeoconnectCreateEditMaps to true to allow the user to create GeoConnect Maps. This boolean effects whether the user sees the map button on the dataset page and if the ingest will create a shape file.

curl -X PUT -d true http://localhost:8080/api/admin/settings/:GeoconnectCreateEditMaps

Set GeoconnectViewMaps to true to allow a user to view existing maps. This boolean effects whether a user will see the "Explore" button.

curl -X PUT -d true http://localhost:8080/api/admin/settings/:GeoconnectViewMaps

v4.2.2

Overview:

v4.2.2 is focused on improving performance of the Dataverse page and Files facet. Additionally, several important support issues were addressed.

For more information see the <a href=https://github.com/IQSS/dataverse/milestones?state=closed>project milestone notes and <a href=https://github.com/IQSS/dataverse/issues?q=milestone%3A%224.2.2+Performance+patch%22+is%3Aclosedd>individual issues

Installation:

Please see our Installation Guide

Upgrades:

If upgrading from v3.6.2 or earlier, please contact us at [email protected] since this requires considerable planning and effort.

If upgrading from v4.x, you will need to follow the upgrade instructions for each intermediate version.

Please note that tabular data tags are indexed in v4.2.2 and that requires a new Solr schema and a complete reindex (see below, under the "post-deployment tasks").

For upgrading from the previous version, v4.2.1 to v4.2.2 do the folllowing:

undeploy the current application:

/glassfish4/bin/asadmin undeploy dataverse-4.2.1

update the solr schema.xml file:
stop solr
remove current schema.xml
replace with the 4.2.2 schema.xml file provided
start solr

deploy the new war file:

/glassfish4/bin/asadmin deploy dataverse-4.2.2.war

run the upgrade psql script provided: upgrade_v4.2.1_to_v4.2.2.sql
restart glassfish

Post-deployment tasks:

1. Reindex Solr

This is a non destructive reindex and can be done while the application is serving users. First clear index time from the dvobject table, by executing the following SQL query:

 update dvobject set indextime=null;

Then from the web server console run:

 curl -X GET "http://localhost:8080/api/admin/index/continue?numPartitions=1&partitionIdToProcess=0"

Alternatively, if you don’t mind doing a destructive reindex, clear the current index:

 curl -X GET “http://localhost:8080/api/admin/index/clear"

Then do an index all:

 curl -X GET "http://localhost:8080/api/admin/index"

Where 8080 is the port where Glassfish is running.

2. The following shell script needs to be run in your files directory:

(the script simply renames any legacy "saved original files", with the filenames like _filename, as filename.orig)

  #!/bin/sh

  find . -type f | while read file
  do
    origfile=`echo $file | sed 's:[^/]*$:_&:'`
    if [ -f "$origfile" ]
    then
       /bin/mv -f $origfile $file".orig" 
    fi
  done

("files directory" is where the Dataverse application is storing physical data files. It is specified as

<jvm-options>-Ddataverse.files.directory=...</jvm-options>

in your domain.xml configuration file)

v4.2.1

Dataverse v4.2.1 is focused on improving performance of the dataset page and system stability by greatly reducing the number database calls made and limiting to 25 the number of files displayed on initialization. In addition, the SWORD API performance was improved to allow retrieval of file listings with large numbers of files.

For more information see the <a href=https://github.com/IQSS/dataverse/milestones?state=closed>project milestone notes and <a href=https://github.com/IQSS/dataverse/issues?q=milestone%3A%224.2.1+Performance+patch%22+is%3Aclosedd>individual issues .

Installation:
Please see our Installation Guide

Upgrades:
If upgrading from v3.6.2 or earlier, please contact us at [email protected] since this requires considerable planning and effort.

If upgrading from v4.x, you will need to follow the upgrade instructions for each intermediate version.

For upgrading from the previous version, v4.2 to v4.2.1 do the folllowing:

1. undeploy the current application
   /glassfish4/bin/asadmin list-applications
   /glassfish4/bin/asdamin undeploy dataverse-4.2
2. deploy the new war file
   /glassfish4/bin/asadmin deploy dataverse-4.2.1.war
   restart glassfish

v4.2

Dataverse v4.2 is focused on improving performance of the files and permissions pages as well as adding batch editing of files.

Numerous bug fixes have been added and new features, including:

• Not displaying the upload error triangle to the user
• Allowing saving of files and terms of use when a required metadata field is not yet completed.
• Ability to set Terms of Use and Terms of Access for a dataset template
• Updates to the responsiveness of the application for smaller screens (including tablets and mobile phones)
• Download all button returns for datasets

For more information see the <a href=https://github.com/IQSS/dataverse/milestones?state=closed>project milestone notes and <a href=https://github.com/IQSS/dataverse/issues?q=milestone%3A4.2+is%3Aclosed>individual issues .

Installation:
Please see our Installation Guide

Upgrades:
If upgrading from v3.6.2 or earlier, please contact us at [email protected] since this requires considerable planning and effort.

If upgrading from v4.x, you will need to follow the upgrade instructions for each intermediate version.

Please note that the added facets in v4.2 require a new Solr schema and a complete reindex. Depending on how much data you have, this could take a while.

For upgrading from the previous version, v4.1 to v4.2 do the folllowing:

1. undeploy the current application
   /glassfish4/bin/asadmin list-applications
   /glassfish4/bin/asamin undeploy dataverse-4.1

2. update the solr schema.xml file
   stop solr
   remove current schema.xml
   replace with the 4.2 schema.xml file provided
   start solr

3. deploy the new war file
   /glassfish4/bin/asadmin deploy dataverse-4.2.war
   run the upgrade psql script provided: upgrade_v4.1_to_v4.2.sql
   restart glassfish

4. Reindex Solr
   This is a non destructive reindex and can be done while the application is serving users. First clear index time from the dvobject table:
   update dvobject set indextime=null;
   Then from the web server console run:
   curl -X GET "http://localhost:8080/api/admin/index/continue?numPartitions=1&partitionIdToProcess=0"

   Alternatively, if you don’t mind doing a destructive reindex, clear the current index:
   curl -X GET “http://localhost:8080/api/admin/index/clear”
   Then do an index all:
   curl -X GET "http://localhost:8080/api/admin/index"

   Where 8080 is the port where Glassfish is running.

v4.1 release

Dataverse v4.1 is a minor release that contains two notable features: My Data and a manage groups UI. Bug fixes include: an updated Two Ravens installer, user guide updates, several API fixes, and a patch to the Grizzly subsystem that was preventing us from fronting Glassfish by Apache.

For more information about changes see the project milestone notes and individual issues.

Installation:

Please see our Installation Guide

Upgrades:

If upgrading from v3.6.2 or earlier, please contact us at [email protected] since this requires considerable planning and effort.

If upgrading from v4.x, you will need to follow the upgrade instructions for each intermediate version.
Please note that the My Data feature requires a new Solr schema and a complete reindex. Depending on how much data you have, this could take a while.

For upgrading from the previous version, v4.0.1 to v4.1 do the folllowing:

1. undeploy the current application
   /glassfish4/bin/asadmin list-applications
   /glassfish4/bin/asamin undeploy dataverse-4.0.1
2. update the solr schema.xml file
   stop solr
   remove current schema.xml
   replace with the 4.1 schema.xml file provided
   start solr
3. deploy the new war file
   /glassfish4/bin/asadmin deploy dataverse-4.1.war
4. run the upgrade psql script provided: upgrade_v4.0.1_to_v4.1.sql
5. restart glassfish
   Note: if you plan on fronting Glassfish with Apache, apply the Grizzly patch
6. Reindex Solr
   This is a destructive reindex. First back up and move the current index files.

Then from the web server console run:
curl -X GET "http://localhost:8080/api/admin/index"

Where 8080 is the port where Glassfish is running.

58a9f33 is v. 4.1 build 71

v4.0.1

Dataverse v4.0.1 is a point release that is mostly about bug fixes and clean up tasks following the v4.0 release.

Most notable is a fix for deleting files. For more information about changes see the project milestone notes and individual issues.

Installation:

Please see our Installation Guide

Upgrades:

If upgrading from v3.6.2 or earlier, please contact us at [email protected] since this requires considerable planning and effort.

If upgrading from v4.0, you will need to do the following:

1. undeploy the current application
   /glassfish4/bin/asadmin list-applications
   /glassfish4/bin/asamin undeploy dataverse-4.0.1
2. update the solr schema.xml file
   stop solr
   remove current schema.xml
   get the 4.0.1 schema.xml file
   https://raw.githubusercontent.com/IQSS/dataverse/4.0.1/conf/solr/4.6.0/schema.xml
   start solr
3. deploy the new war file
   /glassfish4/bin/asadmin deploy dataverse-4.0.1.war
4. run the upgrade psql script https://raw.githubusercontent.com/IQSS/dataverse/4.0.1/scripts/database/upgrades/upgrade_v4.0_to_v4.0.1.sql
5. restart glassfish

4.0

-Latest Release, 4.0.1, can be found here: https://github.com/IQSS/dataverse/releases/tag/v4.0.1

4.0 Release Notes

Dataverse 4.0 is completely rewritten and focused on improving usability, extending support to multiple disciplines, enhanced API support, and an improved permissions model. This list of new features and changes is not exhaustive, see the user guides and GitHub project for more information.

New Features:

• Improved add data and edit workflows.
• Improved restrict file and request access workflows.
• Both online and email-based workflow event notification.
• Improved messaging.
• Customizable search facets for each dataverse.
• Greater use of search facets for data discovery.
• Solr server for faster, more inclusive search results.
• Linked search allows adding search-based results to a dataverse.
• Simplified dataverse customization through themes and widgets.
• Featured dataverses allows highlighting selected dataverses.
• Default CC0 waiver for datasets.
• Support for custom metadata fields
• New search and native APIs.
• Dataverse administrators can create user groups.
• Dataverse administrators can create nested dataverses.
• Support for Rdata ingest.
• New point and click online analysis tool, TwoRavens.
• Shibboleth single sign on support, currently experimental.

Changes:

• Faceting, linked search, and sub dataverses replace collections.
• Extensive metadata domainsreplace controlled vocabulary.
• z39.50 support is ending.
• Widgets replace iframe customization.
• Dataverse widgets built in to OpenScholar
• Dataverse themes replace html header and footer customization.

Technology Upgrade:

• PrimeFaces, with Bootstrap
• Java 7
• Glassfish 4.1

Issues:

• Shibboleth support is experimental.
• In some cases deleting a dataset fails.
• Linked datasets, dataverses, and searches are currently available to super users only.

Installation:

• Please see our installation guide: http://guides.dataverse.org/en/latest/installation/

Upgrading:

• Upgrading from a prior version requires considerable planning and testing. Please contact us for support at [email protected]