How to deal with deleted nSDS storage pools still showing in NexaSphere UI

nexus

In NexaVM nSDS, when a Ceph storage pool is removed from the underlying storage cluster before updating the NexaSphere Management Node, deleting it via the Web UI fails or leaves orphaned entries in the database.

Environment Variables Mapping:
Before executing SQL queries, identify the specific parameters for your environment:
<PRIMARY_STORAGE_UUID>: UUID of the NexaVM nsds Primary Storage resource (found via SELECT uuid FROM PrimaryStorageVO;)
<REMOVED_POOL_NAME>: Name of the removed Ceph pool (e.g., pool-bb59e1eb80f440e08a802be07ae3f6e1).
<ACTIVE_POOL_NAME>: Name of an existing active Ceph pool (e.g., pool-3059cde8dd3a4778b110fbeabe3eb0d0).
<REMOVED_POOL_UUID>: UUID of the CephPrimaryStoragePoolVO record corresponding to the removed pool.

Step 1: Create a Database Backup
SSH into the Management Node and create a safety backup of the MariaDB database:

zstack-ctl dump_mysql

or

mysqldump -u root -p zstack > /root/zstack_db_backup_$(date +%Y%m%d_%H%M%S).sql

Step 2: Clean Orphaned Records in ImageCacheVO
Log in to MySQL/MariaDB on the Management Node:

mysql -u root -p zstack

Verify image cache entries linked to the removed pool:

SELECT * FROM ImageCacheVO WHERE installUrl LIKE '%<REMOVED_POOL_NAME>%';

Delete the orphaned entries:

DELETE FROM ImageCacheVO WHERE installUrl LIKE '%<REMOVED_POOL_NAME>%';

Step 3: Reassign SystemTags to an Active Pool
Do not delete rootVolumePoolName and dataVolumePoolName tags; reassign them to an active pool (<ACTIVE_POOL_NAME>) to prevent NoneType null-pointer exceptions during datastore initialization.

Locate SystemTags associated with the primary storage:

SELECT uuid, resourceUuid, tag FROM SystemTagVO 
WHERE resourceUuid = '<PRIMARY_STORAGE_UUID>' 
  AND (tag LIKE '%rootVolumePoolName%' OR tag LIKE '%dataVolumePoolName%' OR tag LIKE '%imageCachePoolName%');

Reassign SystemTags to the active pool:

UPDATE SystemTagVO 
SET tag = 'ceph::rootVolumePoolName::<ACTIVE_POOL_NAME>' 
WHERE resourceUuid = '<PRIMARY_STORAGE_UUID>' AND tag LIKE 'ceph::rootVolumePoolName::%';

UPDATE SystemTagVO 
SET tag = 'ceph::dataVolumePoolName::<ACTIVE_POOL_NAME>' 
WHERE resourceUuid = '<PRIMARY_STORAGE_UUID>' AND tag LIKE 'ceph::dataVolumePoolName::%';

UPDATE SystemTagVO 
SET tag = 'ceph::default::imageCachePoolName::<ACTIVE_POOL_NAME>' 
WHERE resourceUuid = '<PRIMARY_STORAGE_UUID>' AND tag LIKE 'ceph::default::imageCachePoolName::%';

Step 4: Purge Removed Pool Records from Registration Tables
Delete entries of the removed pool to prevent periodic health checks from failing (SYS.1006 error):

Locate the record of the removed pool:

SELECT uuid, poolName, aliasName, type FROM CephPrimaryStoragePoolVO 
WHERE poolName = '<REMOVED_POOL_NAME>';

Remove the pool record from CephPrimaryStoragePoolVO

DELETE FROM CephPrimaryStoragePoolVO WHERE poolName = '<REMOVED_POOL_NAME>';

Delete the corresponding resource entry from ResourceVO

DELETE FROM ResourceVO WHERE uuid = '<REMOVED_POOL_UUID>';

Step 5: Restart Management Services and Perform Reconnect
Restart management services on both management nodes to reload the storage topology in memory:

zsha2 stop-node && zsha2 start-node

1- Log in to the NexaSphere Web UI.
2- Navigate to Data Storage.
3- Select the affected NexaVM nSDS storage and click Actions > Reconnect.
4- Verify that the storage status changes stably to Connected.