Forums

Home / Forums

You need to log in to create posts and topics. Login · Register

Unable to setup Cluster using PetaSAN 4.0


sandeep.kumar
10 Posts
October 20, 2025, 6:51 pm

Hello Team,

Is any one built a PetaSAN cluster using 4.0 version ?

I'm seeing issue while setting up PetaSAN cluster using 4.0 version, when I try to add 3rd node to the cluster, seeing error "Error Deploying Node" "Error list - Cannot create OSDs for node petasan-03-admin". Please find the log below and suggest.

23/10/2025 16:05:01 INFO Start settings IPs
23/10/2025 16:06:02 ERROR Config file error. The PetaSAN os maybe just installed.
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/PetaSAN/backend/cluster/deploy.py", line 76, in get_node_status
node_name = config.get_node_info().name
File "/usr/lib/python3/dist-packages/PetaSAN/core/cluster/configuration.py", line 99, in get_node_info
with open(config.get_node_info_file_path(), 'r') as f:
FileNotFoundError: [Errno 2] No such file or directory: '/opt/petasan/config/node_info.json'
23/10/2025 16:06:31 INFO Starting node join
23/10/2025 16:07:01 ERROR Timeout exceeded.
<pexpect.pty_spawn.spawn object at 0x7f61ffe61fd0>
command: /usr/bin/scp
args: ['/usr/bin/scp', '-o', 'StrictHostKeyChecking=no', 'root@10.100.168.95:/root/.ssh/id_rsa.pub', '/root/.ssh/id_rsa.pub']
buffer (last 100 chars): b" \r\nPermission denied, please try again.\r\r\n\rroot@10.100.168.95's password: "
before (last 100 chars): b" \r\nPermission denied, please try again.\r\r\n\rroot@10.100.168.95's password: "
after: <class 'pexpect.exceptions.TIMEOUT'>
match: None
match_index: None
exitstatus: None
flag_eof: False
pid: 3232
child_fd: 9
closed: False
timeout: 30
delimiter: <class 'pexpect.exceptions.EOF'>
logfile: None
logfile_read: None
logfile_send: None
maxread: 2000
ignorecase: False
searchwindowsize: None
delaybeforesend: 0.05
delayafterclose: 0.1
delayafterterminate: 0.1
searcher: searcher_re:
0: EOF
23/10/2025 16:07:01 ERROR Error while copying keys or setting password.
23/10/2025 16:07:24 INFO Starting node join
23/10/2025 16:07:54 ERROR Timeout exceeded.
<pexpect.pty_spawn.spawn object at 0x7f61ffe61610>
command: /usr/bin/scp
args: ['/usr/bin/scp', '-o', 'StrictHostKeyChecking=no', 'root@10.100.168.95:/root/.ssh/id_rsa.pub', '/root/.ssh/id_rsa.pub']
buffer (last 100 chars): b" \r\nPermission denied, please try again.\r\r\n\rroot@10.100.168.95's password: "
before (last 100 chars): b" \r\nPermission denied, please try again.\r\r\n\rroot@10.100.168.95's password: "
after: <class 'pexpect.exceptions.TIMEOUT'>
match: None
match_index: None
exitstatus: None
flag_eof: False
pid: 3295
child_fd: 9
closed: False
timeout: 30
delimiter: <class 'pexpect.exceptions.EOF'>
logfile: None
logfile_read: None
logfile_send: None
maxread: 2000
ignorecase: False
searchwindowsize: None
delaybeforesend: 0.05
delayafterclose: 0.1
delayafterterminate: 0.1
searcher: searcher_re:
0: EOF
23/10/2025 16:07:54 ERROR Error while copying keys or setting password.
23/10/2025 19:59:46 ERROR Config file error. The PetaSAN os maybe just installed.
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/PetaSAN/backend/cluster/deploy.py", line 76, in get_node_status
node_name = config.get_node_info().name
File "/usr/lib/python3/dist-packages/PetaSAN/core/cluster/configuration.py", line 99, in get_node_info
with open(config.get_node_info_file_path(), 'r') as f:
FileNotFoundError: [Errno 2] No such file or directory: '/opt/petasan/config/node_info.json'
23/10/2025 20:00:17 INFO Starting node join
23/10/2025 20:00:17 INFO Successfully copied public keys.
23/10/2025 20:00:17 INFO Successfully copied private keys.
23/10/2025 20:00:18 INFO password set successfully.
23/10/2025 20:00:18 INFO Start copying cluster info file.
23/10/2025 20:00:18 INFO Successfully copied cluster info file.
23/10/2025 20:00:18 INFO Start copying services interfaces file.
23/10/2025 20:00:18 INFO Successfully copied services interfaces file.
23/10/2025 20:00:18 INFO Joined cluster petasan-bo3
23/10/2025 20:21:42 INFO Start settings IPs
23/10/2025 20:27:01 ERROR Cluster is not completed, PetaSAN will check node join status.
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/PetaSAN/backend/cluster/deploy.py", line 85, in get_node_status
raise Exception("Cluster is not completed, PetaSAN will check node join status.")
Exception: Cluster is not completed, PetaSAN will check node join status.
23/10/2025 20:27:39 INFO Starting node join
23/10/2025 20:27:39 INFO Successfully copied public keys.
23/10/2025 20:27:39 INFO Successfully copied private keys.
23/10/2025 20:27:39 INFO password set successfully.
23/10/2025 20:27:39 INFO Start copying cluster info file.
23/10/2025 20:27:40 INFO Successfully copied cluster info file.
23/10/2025 20:27:40 INFO Start copying services interfaces file.
23/10/2025 20:27:40 INFO Successfully copied services interfaces file.
23/10/2025 20:27:40 INFO Joined cluster petasan-bo3
23/10/2025 20:41:49 ERROR 400 Bad Request: The browser (or proxy) sent a request that this server could not understand.
23/10/2025 20:41:49 ERROR 400 Bad Request: The browser (or proxy) sent a request that this server could not understand.
23/10/2025 20:41:49 ERROR 400 Bad Request: The browser (or proxy) sent a request that this server could not understand.
23/10/2025 20:41:49 INFO Set node role completed successfully.
23/10/2025 20:41:49 INFO Set node info completed successfully.
23/10/2025 20:42:12 INFO Stopping petasan services on all nodes.
23/10/2025 20:42:12 INFO Stopping all petasan services.
23/10/2025 20:42:12 INFO Stopping PetaSAN services
23/10/2025 20:42:12 INFO ------------------------------------------
23/10/2025 20:42:12 INFO Stopping files sync service
23/10/2025 20:42:12 INFO files_sync.py process is 4182
23/10/2025 20:42:12 INFO files_sync.py process stopped
23/10/2025 20:42:12 INFO Stopping iSCSI service
23/10/2025 20:42:13 INFO iscsi_service.py process is 4184
23/10/2025 20:42:13 INFO iscsi_service.py process stopped
23/10/2025 20:42:13 INFO Stopping Admin service
23/10/2025 20:42:13 INFO admin.py process is 4186
23/10/2025 20:42:13 INFO admin.py process stopped
23/10/2025 20:42:13 INFO Stopping CIFS service
23/10/2025 20:42:13 INFO Stopping NFS service
23/10/2025 20:42:13 INFO Stopping S3 service
23/10/2025 20:42:13 INFO Starting Node Stats Service
23/10/2025 20:42:15 INFO Starting local clean_ceph.
23/10/2025 20:42:15 INFO Starting clean_ceph
23/10/2025 20:42:15 INFO Stopping ceph services
23/10/2025 20:42:15 INFO Start cleaning config files
23/10/2025 20:42:15 INFO Starting ceph services
23/10/2025 20:42:17 INFO Starting local clean_consul.
23/10/2025 20:42:17 INFO Trying to clean Consul on local node
23/10/2025 20:42:17 INFO delete /opt/petasan/config/etc/consul.d
23/10/2025 20:42:17 INFO delete /opt/petasan/config/var/consul
23/10/2025 20:42:17 INFO Trying to clean Consul on 10.100.168.95
23/10/2025 20:42:18 INFO Trying to clean Consul on 10.100.168.96
23/10/2025 20:42:19 INFO cluster_name: petasan-bo3
23/10/2025 20:42:19 INFO local_node_info.name: petasan-03-admin
23/10/2025 20:42:20 INFO Start consul leaders remotely.
23/10/2025 20:42:23 INFO str_start_command: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consul agent -raft-protocol 2 -config-dir /opt/petasan/config/etc/consul.d/server -bind 10.100.50.204 -retry-join 10.100.50.202 -retry-join 10.100.50.203
23/10/2025 20:42:24 INFO Cluster Node {} joined the cluster and is alivepetasan-03-admin
23/10/2025 20:42:24 INFO Cluster Node {} joined the cluster and is alivepetasan-01-admin
23/10/2025 20:42:24 INFO Cluster Node {} joined the cluster and is alivepetasan-02-admin
23/10/2025 20:42:24 INFO Consul leaders are ready
23/10/2025 20:42:34 INFO S3Server : S3 Settings has been saved in Consul
23/10/2025 20:42:34 INFO NFSServer : Changing NFS Settings in Consul
23/10/2025 20:42:34 INFO NFSServer : NFS Settings has been changed in Consul
23/10/2025 20:42:34 INFO Success saving application config
23/10/2025 20:42:34 INFO Success saving application config
23/10/2025 20:42:35 INFO First monitor started successfully
23/10/2025 20:42:35 INFO create_mgr() fresh install
23/10/2025 20:42:35 INFO create_mgr() started
23/10/2025 20:42:35 INFO create_mgr() cmd : mkdir -p /var/lib/ceph/mgr/ceph-petasan-03-admin
23/10/2025 20:42:35 INFO create_mgr() cmd : ceph --cluster ceph auth get-or-create mgr.petasan-03-admin mon 'allow profile mgr' osd 'allow *' mds 'allow *' -o /var/lib/ceph/mgr/ceph-petasan-03-admin/keyring
23/10/2025 20:42:37 INFO create_mgr() ended successfully
23/10/2025 20:42:37 INFO create_mds() started
23/10/2025 20:42:40 INFO create_mds() cmd : mkdir -p /var/lib/ceph/mds/ceph-petasan-03-admin
23/10/2025 20:42:41 INFO create_mds() ended successfully

23/10/2025 20:42:41 INFO Starting to deploy remote monitors
23/10/2025 20:43:11 INFO Ceph monitors are ready.
23/10/2025 20:43:19 INFO Start deploying ceph OSDs.
23/10/2025 20:43:19 INFO Start cleaning : sda
23/10/2025 20:43:20 INFO Executing : wipefs --all /dev/sda
23/10/2025 20:43:20 INFO Executing : dd if=/dev/zero of=/dev/sda bs=1M count=20 oflag=direct,dsync >/dev/null 2>&1
23/10/2025 20:43:21 INFO Executing : parted -s /dev/sda mklabel gpt
23/10/2025 20:43:21 INFO Executing : partprobe /dev/sda
23/10/2025 20:43:24 INFO Start cleaning : sda
23/10/2025 20:43:24 INFO Executing : wipefs --all /dev/sda
23/10/2025 20:43:25 INFO Executing : dd if=/dev/zero of=/dev/sda bs=1M count=20 oflag=direct,dsync >/dev/null 2>&1
23/10/2025 20:43:25 INFO Executing : parted -s /dev/sda mklabel gpt
23/10/2025 20:43:25 INFO Executing : partprobe /dev/sda
23/10/2025 20:43:30 INFO Start cleaning : sdaa
23/10/2025 20:43:30 INFO Executing : wipefs --all /dev/sdaa
23/10/2025 20:43:30 INFO Executing : dd if=/dev/zero of=/dev/sdaa bs=1M count=20 oflag=direct,dsync >/dev/null 2>&1
23/10/2025 20:43:31 INFO Executing : parted -s /dev/sdaa mklabel gpt
23/10/2025 20:43:31 INFO Executing : partprobe /dev/sdaa
23/10/2025 20:43:34 INFO Creating cache partition num 1 size 51200MB on /dev/sdaa
23/10/2025 20:43:35 INFO Calling partprobe on sdaa device
23/10/2025 20:43:35 INFO Executing partprobe /dev/sdaa
23/10/2025 20:43:35 INFO Calling udevadm on sdaa device
23/10/2025 20:43:35 INFO Executing udevadm settle --timeout 30
23/10/2025 20:43:38 INFO Creating cache partition num 2 size 51200MB on /dev/sdaa
23/10/2025 20:43:39 INFO Calling partprobe on sdaa device
23/10/2025 20:43:39 INFO Executing partprobe /dev/sdaa
23/10/2025 20:43:39 INFO Calling udevadm on sdaa device
23/10/2025 20:43:39 INFO Executing udevadm settle --timeout 30
23/10/2025 20:43:43 INFO Creating cache partition num 3 size 51200MB on /dev/sdaa
23/10/2025 20:43:44 INFO Calling partprobe on sdaa device
23/10/2025 20:43:44 INFO Executing partprobe /dev/sdaa
23/10/2025 20:43:44 INFO Calling udevadm on sdaa device
23/10/2025 20:43:44 INFO Executing udevadm settle --timeout 30
23/10/2025 20:43:47 INFO Creating cache partition num 4 size 51200MB on /dev/sdaa
23/10/2025 20:43:48 INFO Calling partprobe on sdaa device
23/10/2025 20:43:48 INFO Executing partprobe /dev/sdaa
23/10/2025 20:43:48 INFO Calling udevadm on sdaa device
23/10/2025 20:43:48 INFO Executing udevadm settle --timeout 30
23/10/2025 20:43:52 INFO Creating cache partition num 5 size 51200MB on /dev/sdaa
23/10/2025 20:43:53 INFO Calling partprobe on sdaa device
23/10/2025 20:43:53 INFO Executing partprobe /dev/sdaa
23/10/2025 20:43:53 INFO Calling udevadm on sdaa device
23/10/2025 20:43:53 INFO Executing udevadm settle --timeout 30
23/10/2025 20:43:56 INFO Creating cache partition num 6 size 51200MB on /dev/sdaa
23/10/2025 20:43:57 INFO Calling partprobe on sdaa device
23/10/2025 20:43:57 INFO Executing partprobe /dev/sdaa
23/10/2025 20:43:57 INFO Calling udevadm on sdaa device
23/10/2025 20:43:57 INFO Executing udevadm settle --timeout 30
23/10/2025 20:44:01 INFO Creating cache partition num 7 size 51200MB on /dev/sdaa
23/10/2025 20:44:02 INFO Calling partprobe on sdaa device
23/10/2025 20:44:02 INFO Executing partprobe /dev/sdaa
23/10/2025 20:44:02 INFO Calling udevadm on sdaa device
23/10/2025 20:44:02 INFO Executing udevadm settle --timeout 30
23/10/2025 20:44:05 INFO Creating cache partition num 8 size 51200MB on /dev/sdaa
23/10/2025 20:44:07 INFO Calling partprobe on sdaa device
23/10/2025 20:44:07 INFO Executing partprobe /dev/sdaa
23/10/2025 20:44:07 INFO Calling udevadm on sdaa device
23/10/2025 20:44:07 INFO Executing udevadm settle --timeout 30
23/10/2025 20:44:10 INFO Start cleaning : sdb
23/10/2025 20:44:11 INFO Executing : wipefs --all /dev/sdb
23/10/2025 20:44:11 INFO Executing : dd if=/dev/zero of=/dev/sdb bs=1M count=20 oflag=direct,dsync >/dev/null 2>&1
23/10/2025 20:44:11 INFO Executing : parted -s /dev/sdb mklabel gpt
23/10/2025 20:44:12 INFO Executing : partprobe /dev/sdb
23/10/2025 20:44:17 INFO Start cleaning : sda
23/10/2025 20:44:21 INFO Executing : wipefs --all /dev/sda1
23/10/2025 20:44:21 INFO Executing : dd if=/dev/zero of=/dev/sda1 bs=1M count=20 oflag=direct,dsync >/dev/null 2>&1
23/10/2025 20:44:22 INFO Executing : wipefs --all /dev/sda
23/10/2025 20:44:22 INFO Executing : dd if=/dev/zero of=/dev/sda bs=1M count=20 oflag=direct,dsync >/dev/null 2>&1
23/10/2025 20:44:22 INFO Executing : parted -s /dev/sda mklabel gpt
23/10/2025 20:44:22 INFO Executing : partprobe /dev/sda
23/10/2025 20:44:27 INFO ==================================================
23/10/2025 20:44:27 INFO ===== Building DM-Writecache =====
23/10/2025 20:44:27 INFO ==================================================
23/10/2025 20:44:27 INFO Step 1 : Preparing Slow Disk :
23/10/2025 20:44:27 INFO ------------------------------
23/10/2025 20:44:28 INFO Creating data partition num 1 size 11211776MB on /dev/sdb
23/10/2025 20:44:29 INFO Calling partprobe on sdb device
23/10/2025 20:44:29 INFO Executing partprobe /dev/sdb
23/10/2025 20:44:29 INFO Calling udevadm on sdb device
23/10/2025 20:44:29 INFO Executing udevadm settle --timeout 30
23/10/2025 20:44:32 INFO origin_part_name = /dev/sdb1
23/10/2025 20:44:32 INFO done

23/10/2025 20:44:32 INFO Step 2 : Preparing Fast Disk :
23/10/2025 20:44:32 INFO ------------------------------
23/10/2025 20:44:32 INFO Start cleaning : sdaa1
23/10/2025 20:44:32 INFO Executing : umount /dev/sdaa1
23/10/2025 20:44:32 ERROR Error executing : umount /dev/sdaa1
23/10/2025 20:44:32 INFO Executing : wipefs --all /dev/sdaa1
23/10/2025 20:44:32 INFO Executing : dd if=/dev/zero of=/dev/sdaa1 bs=1M count=20 oflag=direct,dsync >/dev/null 2>&1
23/10/2025 20:44:32 INFO Executing : dd bs=1M seek=197607424 if=/dev/zero of=/dev/sdaa1 count=20 oflag=seek_bytes,direct,dsync >/dev/null 2>&1
23/10/2025 20:44:32 INFO cache_part_path = /dev/sdaa1
23/10/2025 20:44:32 INFO done

23/10/2025 20:44:32 INFO Step 3 : Creating Physical Volumes :
23/10/2025 20:44:32 INFO ------------------------------------
23/10/2025 20:44:33 INFO done

23/10/2025 20:44:33 INFO Step 4 : Creating Volume Group :
23/10/2025 20:44:33 INFO --------------------------------
23/10/2025 20:44:34 INFO Start remove osd.0 from crush map
23/10/2025 20:44:37 INFO osd.0 is removed from crush map
23/10/2025 20:44:37 INFO new next_osd_id = 0

23/10/2025 20:44:37 INFO new vg_name = ps-08188e70-22b6-4769-8191-b3eb27583860-wc-osd.0

23/10/2025 20:44:37 INFO vg_name = ps-08188e70-22b6-4769-8191-b3eb27583860-wc-osd.0
23/10/2025 20:44:37 INFO done

23/10/2025 20:44:37 INFO Step 5 : Creating Logical Volumes :
23/10/2025 20:44:37 INFO -----------------------------------
23/10/2025 20:44:38 INFO main_lv_name = main
23/10/2025 20:44:38 INFO cache_lv_name = cache
23/10/2025 20:44:38 INFO done

23/10/2025 20:44:38 INFO Step 6 : Building dm-writecache :
23/10/2025 20:44:38 INFO ---------------------------------
23/10/2025 20:44:38 INFO cmd = lvchange -a n ps-08188e70-22b6-4769-8191-b3eb27583860-wc-osd.0/cache ---> done
23/10/2025 20:44:38 INFO cmd = modprobe dm-writecache ---> done
23/10/2025 20:44:46 INFO cmd = lvconvert --yes --type writecache --cachevol cache --cachesettings 'writeback_jobs=15360 block_size=4096' ps-08188e70-22b6-4769-8191-b3eb27583860-wc-osd.0/main ---> done
23/10/2025 20:44:46 INFO Start setting partition type for sdaa1
23/10/2025 20:44:46 INFO Starting sgdisk -t 1:f7e357a6-a096-4498-93c1-0ff812402190 /dev/sdaa
23/10/2025 20:44:47 INFO Calling partprobe on sdaa device
23/10/2025 20:44:47 INFO Executing partprobe /dev/sdaa
23/10/2025 20:44:47 INFO Calling udevadm on sdaa device
23/10/2025 20:44:47 INFO Executing udevadm settle --timeout 30
23/10/2025 20:44:51 INFO lv_path = ps-08188e70-22b6-4769-8191-b3eb27583860-wc-osd.0/main
23/10/2025 20:44:51 INFO running writecache tuning script
23/10/2025 20:44:54 INFO done
23/10/2025 20:44:54 INFO ==================================================
23/10/2025 20:44:54 INFO Start prepare bluestore OSD : ps-08188e70-22b6-4769-8191-b3eb27583860-wc-osd.0/main
23/10/2025 20:44:54 INFO OSD_ID = 0
23/10/2025 20:44:55 INFO Start remove osd.0 from crush map
23/10/2025 20:44:58 INFO osd.0 is removed from crush map
23/10/2025 20:44:58 INFO creating new journal partition.
23/10/2025 20:44:59 INFO Creating journal-db-0-08188e70 partition num 1 size 11211776MB on /dev/sda
23/10/2025 20:45:00 INFO Calling partprobe on sda device
23/10/2025 20:45:00 INFO Executing partprobe /dev/sda
23/10/2025 20:45:00 INFO Calling udevadm on sda device
23/10/2025 20:45:00 INFO Executing udevadm settle --timeout 30
23/10/2025 20:45:03 INFO Starting : ceph-volume --log-path /opt/petasan/log lvm prepare --bluestore --data ps-08188e70-22b6-4769-8191-b3eb27583860-wc-osd.0/main --osd-id 0 --block.db /dev/sda1
23/10/2025 20:45:22 INFO starting activate osd.
23/10/2025 20:45:23 INFO Starting : ceph-volume --log-path /opt/petasan/log lvm activate --bluestore 0 cdefd6ed-e21d-4fcf-a7d3-0c21ba2ded58
23/10/2025 20:45:24 INFO Successfully executed ceph-volume prepare for sdb
23/10/2025 20:45:24 INFO Start cleaning : sdc
23/10/2025 20:45:25 INFO Executing : wipefs --all /dev/sdc
23/10/2025 20:45:26 INFO Executing : dd if=/dev/zero of=/dev/sdc bs=1M count=20 oflag=direct,dsync >/dev/null 2>&1
23/10/2025 20:45:26 INFO Executing : parted -s /dev/sdc mklabel gpt
23/10/2025 20:45:26 INFO Executing : partprobe /dev/sdc
23/10/2025 20:45:34 INFO ==================================================
23/10/2025 20:45:34 INFO ===== Building DM-Writecache =====
23/10/2025 20:45:34 INFO ==================================================
23/10/2025 20:45:34 INFO Step 1 : Preparing Slow Disk :
23/10/2025 20:45:34 INFO ------------------------------
23/10/2025 20:45:34 INFO Creating data partition num 1 size 11211776MB on /dev/sdc
23/10/2025 20:45:36 INFO Calling partprobe on sdc device
23/10/2025 20:45:36 INFO Executing partprobe /dev/sdc
23/10/2025 20:45:36 INFO Calling udevadm on sdc device
23/10/2025 20:45:36 INFO Executing udevadm settle --timeout 30
23/10/2025 20:45:39 INFO origin_part_name = /dev/sdc1
23/10/2025 20:45:39 INFO done

23/10/2025 20:45:39 INFO Step 2 : Preparing Fast Disk :
23/10/2025 20:45:39 INFO ------------------------------
23/10/2025 20:45:39 INFO Start cleaning : sdaa2
23/10/2025 20:45:39 INFO Executing : umount /dev/sdaa2
23/10/2025 20:45:39 ERROR Error executing : umount /dev/sdaa2
23/10/2025 20:45:39 INFO Executing : wipefs --all /dev/sdaa2
23/10/2025 20:45:39 INFO Executing : dd if=/dev/zero of=/dev/sdaa2 bs=1M count=20 oflag=direct,dsync >/dev/null 2>&1
23/10/2025 20:45:39 INFO Executing : dd bs=1M seek=197607424 if=/dev/zero of=/dev/sdaa2 count=20 oflag=seek_bytes,direct,dsync >/dev/null 2>&1
23/10/2025 20:45:39 INFO cache_part_path = /dev/sdaa2
23/10/2025 20:45:39 INFO done

23/10/2025 20:45:39 INFO Step 3 : Creating Physical Volumes :
23/10/2025 20:45:39 INFO ------------------------------------
23/10/2025 20:45:40 INFO done

23/10/2025 20:45:40 INFO Step 4 : Creating Volume Group :
23/10/2025 20:45:40 INFO --------------------------------
23/10/2025 20:45:40 INFO Start remove osd.1 from crush map
23/10/2025 20:45:43 INFO osd.1 is removed from crush map
23/10/2025 20:45:43 INFO new next_osd_id = 1

23/10/2025 20:45:43 INFO new vg_name = ps-08188e70-22b6-4769-8191-b3eb27583860-wc-osd.1

23/10/2025 20:45:44 INFO vg_name = ps-08188e70-22b6-4769-8191-b3eb27583860-wc-osd.1
23/10/2025 20:45:44 INFO done

23/10/2025 20:45:44 INFO Step 5 : Creating Logical Volumes :
23/10/2025 20:45:44 INFO -----------------------------------
23/10/2025 20:45:44 INFO main_lv_name = main
23/10/2025 20:45:44 INFO cache_lv_name = cache
23/10/2025 20:45:44 INFO done

23/10/2025 20:45:44 INFO Step 6 : Building dm-writecache :
23/10/2025 20:45:44 INFO ---------------------------------
23/10/2025 20:45:44 INFO cmd = lvchange -a n ps-08188e70-22b6-4769-8191-b3eb27583860-wc-osd.1/cache ---> done
23/10/2025 20:45:44 INFO cmd = modprobe dm-writecache ---> done
23/10/2025 20:45:53 INFO cmd = lvconvert --yes --type writecache --cachevol cache --cachesettings 'writeback_jobs=15360 block_size=4096' ps-08188e70-22b6-4769-8191-b3eb27583860-wc-osd.1/main ---> done
23/10/2025 20:45:53 INFO Start setting partition type for sdaa2
23/10/2025 20:45:53 INFO Starting sgdisk -t 2:f7e357a6-a096-4498-93c1-0ff812402190 /dev/sdaa
23/10/2025 20:45:54 INFO Calling partprobe on sdaa device
23/10/2025 20:45:54 INFO Executing partprobe /dev/sdaa
23/10/2025 20:45:54 INFO Calling udevadm on sdaa device
23/10/2025 20:45:54 INFO Executing udevadm settle --timeout 30
23/10/2025 20:45:57 INFO lv_path = ps-08188e70-22b6-4769-8191-b3eb27583860-wc-osd.1/main
23/10/2025 20:45:57 INFO running writecache tuning script
23/10/2025 20:46:01 INFO done
23/10/2025 20:46:01 INFO ==================================================
23/10/2025 20:46:01 INFO Start prepare bluestore OSD : ps-08188e70-22b6-4769-8191-b3eb27583860-wc-osd.1/main
23/10/2025 20:46:01 INFO OSD_ID = 1
23/10/2025 20:46:01 INFO Start remove osd.1 from crush map
23/10/2025 20:46:04 INFO osd.1 is removed from crush map
23/10/2025 20:46:04 INFO creating new journal partition.
23/10/2025 20:46:05 INFO Creating journal-db-1-08188e70 partition num 2 size 11211776MB on /dev/sda
23/10/2025 20:46:06 INFO Calling partprobe on sda device
23/10/2025 20:46:06 INFO Executing partprobe /dev/sda
23/10/2025 20:46:06 INFO Calling udevadm on sda device
23/10/2025 20:46:06 INFO Executing udevadm settle --timeout 30
23/10/2025 20:46:09 INFO Starting : ceph-volume --log-path /opt/petasan/log lvm prepare --bluestore --data ps-08188e70-22b6-4769-8191-b3eb27583860-wc-osd.1/main --osd-id 1 --block.db /dev/sda2
23/10/2025 20:46:27 INFO starting activate osd.
23/10/2025 20:46:29 INFO Starting : ceph-volume --log-path /opt/petasan/log lvm activate --bluestore 1 b6acfb28-1db5-404c-9be5-dc1bf3b34a52
23/10/2025 20:46:30 INFO Successfully executed ceph-volume prepare for sdc
23/10/2025 20:46:30 INFO Start cleaning : sdd
23/10/2025 20:46:31 INFO Executing : wipefs --all /dev/sdd
23/10/2025 20:46:31 INFO Executing : dd if=/dev/zero of=/dev/sdd bs=1M count=20 oflag=direct,dsync >/dev/null 2>&1
23/10/2025 20:46:31 INFO Executing : parted -s /dev/sdd mklabel gpt
23/10/2025 20:46:32 INFO Executing : partprobe /dev/sdd
23/10/2025 20:46:37 ERROR Could not build ceph OSDs.
23/10/2025 20:46:37 ERROR ['scripts_create_osd_disk_failure_node%petasan-03-admin']

#1

admin
3,054 Posts
October 21, 2025, 5:54 pm

check you hardware, specifically node 3 disks. it looks like the 2nd or 3rd OSD failed creation.

you may get more logs in

/opt/petasan/log/ceph-volume.log

to have more control, you can build the cluster first without adding OSDs, the add them manually one by one once cluster build, this may help you identify specific faulty disk and will not abort the cluster creation process.

#2