Friday, August 13, 2021

Liferay DXP Database Upgrade from 7.0 to 7.3

Step 1. Prepare for migration

Before migrating the existing portal to 7.3 it’s recommended to remove unnecessary files and objects

·      Remove unused objects

It’s good practice to find not required objects that can be deleted by examining the tables that were large or contained lots of records. Below listed down some of the common entities to find unneeded objects.

o   Sites: Remove unused sites and its content

o   Instances: Remove instance which is no longer needed. Since they consume highest object in the hierarchy, removing it can optimize upgradation process.

o   Web Content: Liferay generates a new version whenever we are updating any web content. You can delete all older versions of web content.

o   Users: Remove all inactive users and its related data to optimize the process.

o   Layouts: Remove unnecessary layouts.

 

·      Remove duplicated structure names

For LR 6.2 you can have different web content structures in your portal, they are likely to have fields with the same names. But DXP requires unique field names of fields for structures. Therefore, we need to rename structure to have a unique field name.After removing unused objects from your portal, make sure to test those in your older environment. In that case, you can more easily find out any issues, knowing that they’re not related to upgrade processes.

 

Step 2: Liferay Database migration

·       In this step, we will migrate our database from LR 7.0 to LR 7.3. After removing unused objects now our portal is ready to migrate DB from 7.0 to 7.3/DXP.

·       Liferay provides a database upgrade tool that will migrate our database.

·       Before migrating existing DB to LR DXP make sure that you have a backup of it.

Follow below steps :-

 

1.     Create a new database for Liferay 7.3/DXP in Oracle server

e.g. create database Liferay_dxp_db character set utf8

2.     Take backup and restore

Take dump from liferay 7.0 database and restore that dump into a new liferay 7.3/DXP database

Connect to database

3.    Copy data

Copy document_library (You can find from [Liferay Home]/data/) folder from Liferay 7.0 to Liferay 7.3/DXP, copy images folders into data folder also. This data will be stored in the default store and you can also choose your location, it's up to your choice.

 

4.    Configure Your Documents and Media File Store

If the document_library_path is configured in different location then

a)  Create a file with the name

com.liferay.portal.store.file.system.configuration.FileSystemStoreConfiguration.config  under [Liferay_HOME]/osgi/configs folder

b) configure the same path to “rootDir” property in Simple File Store:    

rootDir="C:/dev/project/Liferay_7_3_dev/data/document_library"

 

5.    Disable indexer and autoUpgrade

Now, we need to disable indexing since we are going to start the database upgrade process for our new environment. Using this we can disable reindexing content and performance issues to be arised. For this, we have to follow below steps:-

·       Need to create file in liferay 7.3 DXP :

"com.liferay.portal.search.configuration.IndexStatusManagerConfiguration.config" on this location [Liferay Home]/osgi/configs and add below content

indexReadOnly="true"

·       Then create another file in the same folder :

"com.liferay.portal.upgrade.internal.configuration.ReleaseManagerConfiguration.config" and add below content

autoUpgrade="false"

6.     Configure the upgrade tool

Liferay 7.x provides a dedicated tool for database upgrade 'portal-tools-db-upgrade-client' in [Liferay Home]/tools folder

·      Tuning Database upgrade process:

Make sure to provide adequate memory for the database upgrade tool’s Java process. Also make sure to set the file encoding to UTF-8 and the time zone to GMT. Here are the Java process settings:

  Open db_upgrade.bat file and edit this line:

 java -jar com.liferay.portal.tools.db.upgrade.client.jar  -j   “ -Dfile.encoding=UTF-8

  -Duser.timezone=GMT -Xmx8000m”

 

Now, we need to configure below files before running the upgrade in [Liferay Home]/tools folder:

·      app-server.properties

    dir=../../tomcat-9.0.40

    extra.lib.dirs=/bin

    global.lib.dir=/lib

    portal.dir=/webapps/ROOT

    server.detector.server.id=tomcat

·      portal-upgrade-database.properties(mention database name of them dump taken )

jdbc.default.driverClassName=oracle.jdbc.OracleDriver

jdbc.default.url=jdbc:oracle:thin:@host:1521:db

jdbc.default.username=XXXXXX

jdbc.default.password=xxxxxx

·      portal-upgrade-ext.properties

liferay.home=../../

 

module.framework.base.dir=../../osgi

hibernate.jdbc.batch_size=0

json.deserialization.whitelist.class.names=\

com.liferay.portal.kernel.service.permission.ModelPermissions,\

com.liferay.portal.kernel.internal.service.permission.ModelPermissionsImpl,\

com.liferay.portal.kernel.service.ServiceContext,\

sun.util.calendar.ZoneInfo

dl.store.impl=com.liferay.portal.store.file.system.FileSystemStore(optional)

 

 

7.     Verify buildNumber

 We have to check a release buildNumber from DB before running the upgrade tool, it will be incremented after each deployment in Service Builder which is used for generating internal database persistence code.

Using this query you can find current buildNumber as 7010:

select buildnumber from Release_ where servletcontextname = 'portal';

Make sure below steps before running the db_upgrade.bat file,

·      Place ojdbc6.jar in /lib/ext folder and stop the any running liferay server

·      Since we are using an Oracle database, it is highly recommend putting this property hibernate.jdbc.batch_size=0 in portal-upgrade-ext.properties. This prevents issues when copying or writing CLOBs or other large data during the upgrade.

·      Verify if the database user has full read and write permissions to the Liferay 7.0 database

·      Verify if gogo shell is already running or not? If running kill the task with process id.

C:\Users\VKOTESW>netstat -ano | findstr :9201

C:\Users\VKOTESW>taskkill /PID 10180 /F

 

·      Whenever an upgrade fails, it is best practice to restore the database back to it's pre-upgraded state. This is because if an upgrade fails, there is a very high chance of corrupted data; and re-running the upgrade without restoring it may not fix that.

·       

Run upgrade tool using below command (Path [Liferay Home]/tools/portal-tools-db-upgrade-client folder) :

For window

Double click on “db_upgrade.bat” file

 

The upgrade failed due to a missing database index

com.liferay.dynamic.data.mapping.internal.upgrade.v3_1_0.UpgradeDDMStructureLayout:

This would indicate that the upgrade is trying to drop IX_B7158C0A, but it does not exist. Creating the IX_B7158C0A index will likely resolve this error and allow the com.liferay.dynamic.data.mapping.service to succeed.

https://github.com/liferay/liferay-portal/blob/master/modules/apps/dynamic-data-mapping/dynamic-data-mapping-service/src/main/resources/com/liferay/dynamic/data/mapping/internal/upgrade/v1_0_0/dependencies/update.sql#L52

Execute below script on Liferay 7.0 database fresh dump and run the db_upgrade.bat again

create unique index IX_B7158C0A on DDMStructureLayout (structureVersionId);

One possible way to have the portal create it in the database would be to set database.indexes.update.on.startup=true in the portal-ext.properties file of your Liferay 7.0, start up the bundle and shut it down again once fully started. Then use a copy of this 7.0 database once it is confirmed that IX_B7158C0A exists for another upgrade attempt to 7.3.

·      NOTE: Whenever an upgrade fails, it is best practice to restore the database back to it's pre-upgraded state. This is because if an upgrade fails, there is a very high chance of corrupted data; and re-running the upgrade without restoring it may not fix that.

·      Even if the schema version states 7310, if an error like this occurred, then the upgrade will not be considered successful and the upgrade will need to be done again.           

    Your upgrade if failed it will open gogo shell execute below command to complete    upgrade all remaining upgrade:

>upgrade:executeAll

 

After upgrade again check build number it should be 7310

8.     Enabling indexer

Now, The last and most important step is remaining which can only be taken post-upgrade and to enable indexer. For this, you have to modify the file "com.liferay.portal.search.configuration.IndexStatusManagerConfiguration.config" in the folder [Liferay Home]/osgi/configs with the following content

indexReadOnly="false"

 

Liferay Core is been upgraded now you can  start your portal and check !!

COMMON ERRORS:

Though the property database.indexes.update.on.startup=true exist in the portal-ext.properties file some times indexs will not create. So we created the index manually with the name IX_B7158C0A on DDMStructureLayout in database and ran the upgrade on failed database only. Still we see some issue in the upgrade log for the below module,

2021-07-15 08:06:57.472 INFO [main][UpgradeProcess:115] Failed upgrade process com.liferay.journal.internal.upgrade.v1_1_0.UpgradeJournalArticleLocalizedValues in 7 ms

(1213)[com.liferay.portal.upgrade.internal.release.ReleaseManagerImpl(3102)] : The activate method has thrown an exception

com.liferay.portal.kernel.upgrade.UpgradeException: java.lang.IllegalStateException: JournalArticle must have title and description columns

it is being thrown from this code https://github.com/liferay/liferay-portal/blob/7.3.x/modules/apps/journal/journal-service/src/main/java/com/liferay/journal/internal/upgrade/v1_1_0/UpgradeJournalArticleLocalizedValues.java#L67 . This would suggest that the Upgrade Tool is unable to access the title and description columns from the JournalArticleLocalization table. JournalArticleLocalization is created in Liferay 7.1, and should have been created during the upgrade process since you're coming from 7.0.

Solution: Restored the database back to it's pre-upgraded state and start the upgrade with fresh setup.

Note: Whenever an upgrade fails, it is best practice to restore the database back to it's pre-upgraded state. This is because if an upgrade fails, there is a very high chance of corrupted data; and re-running the upgrade without restoring it may not fix that. Before you run the next upgrade attempt, please restore it back to 7.0. Make sure to create IX_B7158C0A as noted in the Github link.

2021-07-29 06:46:08.798 WARN  [main][LiferayJSONDeserializationWhitelist:50] Unable to deserialize sun.util.calendar.ZoneInfo due to security restrictions

2021-07-29 06:01:52.628 WARN  [main][LiferayJSONDeserializationWhitelist:50] Unable to deserialize com.liferay.portal.kernel.service.permission.ModelPermissions due to security restrictions

Solution: If we get above WARN messages then add the below property in portal-upgrade-ext.properties and start upgrade again

json.deserialization.whitelist.class.names=\

com.liferay.portal.kernel.service.permission.ModelPermissions,\

com.liferay.portal.kernel.internal.service.permission.ModelPermissionsImpl,\

com.liferay.portal.kernel.service.ServiceContext,\

sun.util.calendar.ZoneInfo

 

 

 

No comments:

Post a Comment