Failed Dependency on setting up network
podilarius
44 Posts
December 23, 2021, 6:02 pmQuote from podilarius on December 23, 2021, 6:02 pmAfter installation of a fresh install of 3.0.1 and cluster joins, I restarted node 1 for reboot test and network will not start unless I start it manually.
the python links are good, but on start up, I get:
syslog:Dec 23 17:48:12 prod-dbnk-cphsvr1 systemd[1]: Dependency failed for ifup for eth2.7.
syslog:Dec 23 17:48:12 prod-dbnk-cphsvr1 systemd[1]: ifup@eth2.7.service: Job ifup@eth2.7.service/start failed with result 'dependency'.
If I complete this cycle on console, it all starts up.
systemctl restart networking
systemctl restart petasan*
Let me know if I can provide anything.
After installation of a fresh install of 3.0.1 and cluster joins, I restarted node 1 for reboot test and network will not start unless I start it manually.
the python links are good, but on start up, I get:
syslog:Dec 23 17:48:12 prod-dbnk-cphsvr1 systemd[1]: Dependency failed for ifup for eth2.7.
syslog:Dec 23 17:48:12 prod-dbnk-cphsvr1 systemd[1]: ifup@eth2.7.service: Job ifup@eth2.7.service/start failed with result 'dependency'.
If I complete this cycle on console, it all starts up.
systemctl restart networking
systemctl restart petasan*
Let me know if I can provide anything.
admin
2,918 Posts
December 23, 2021, 8:47 pmQuote from admin on December 23, 2021, 8:47 pm1) Double check your hardware/connections/ip configuration..etc
2) If still you have issue, first thing after reboot, please post:
ip addr list
/opt/petasan/config/cluster_info.json
/opt/petasan/config/node_info.json
/opt/petasan/log/PetaSAN.log
if you are not able to ssh due to no ips at all, add ip via
ip addr add
instead of restarting any services, then scp the above
1) Double check your hardware/connections/ip configuration..etc
2) If still you have issue, first thing after reboot, please post:
ip addr list
/opt/petasan/config/cluster_info.json
/opt/petasan/config/node_info.json
/opt/petasan/log/PetaSAN.log
if you are not able to ssh due to no ips at all, add ip via
ip addr add
instead of restarting any services, then scp the above
Last edited on December 23, 2021, 8:48 pm by admin · #2
podilarius
44 Posts
December 23, 2021, 9:34 pmQuote from podilarius on December 23, 2021, 9:34 pm
- It was fine rebooting until I did ' apt update ; apt upgrade -y '.
- I have it up, I will send that when I get back to working on that system.
In the mean time, what is the best way to post file in here?
- It was fine rebooting until I did ' apt update ; apt upgrade -y '.
- I have it up, I will send that when I get back to working on that system.
In the mean time, what is the best way to post file in here?
admin
2,918 Posts
December 23, 2021, 10:56 pmQuote from admin on December 23, 2021, 10:56 pmcan you also show
dpkg -l
you can share files on cloud storage (dropbox/google drive..) and post url
can you also show
dpkg -l
you can share files on cloud storage (dropbox/google drive..) and post url
podilarius
44 Posts
December 28, 2021, 11:34 pmQuote from podilarius on December 28, 2021, 11:34 pmHere is a link to the files you are requesting. I will add the dpkg now.
https://drive.google.com/drive/folders/1UWsFAVKHo8ePqceDk38F_Es-47OElr6U?usp=sharing
Here is a link to the files you are requesting. I will add the dpkg now.
https://drive.google.com/drive/folders/1UWsFAVKHo8ePqceDk38F_Es-47OElr6U?usp=sharing
Last edited on December 28, 2021, 11:34 pm by podilarius · #5
podilarius
44 Posts
December 30, 2021, 4:52 pmQuote from podilarius on December 30, 2021, 4:52 pmCould this also be a function of unsynced time?
Since I corrected the clocks, everything seems to be working as advertised.
Could this also be a function of unsynced time?
Since I corrected the clocks, everything seems to be working as advertised.
admin
2,918 Posts
December 31, 2021, 12:18 pmQuote from admin on December 31, 2021, 12:18 pmIt was fine rebooting until I did ' apt update ; apt upgrade -y '.
do you mean you got errors on your boot screen ? kernel errors / systemd errors ? network interfaces not up ? or was this just the Ceph cluster was not healthy ? if the later then yes the clock sync is important specially when you reboot multiple nodes at the same time, the cluster cannot get agreement on who has latest data, If the time sync among booting nodes does not correct itself quickly, make sure you hardware clock on your motherboards are all working correctly.
It was fine rebooting until I did ' apt update ; apt upgrade -y '.
do you mean you got errors on your boot screen ? kernel errors / systemd errors ? network interfaces not up ? or was this just the Ceph cluster was not healthy ? if the later then yes the clock sync is important specially when you reboot multiple nodes at the same time, the cluster cannot get agreement on who has latest data, If the time sync among booting nodes does not correct itself quickly, make sure you hardware clock on your motherboards are all working correctly.
podilarius
44 Posts
December 31, 2021, 6:57 pmQuote from podilarius on December 31, 2021, 6:57 pmNo errors on boot screen that I could see. Just the error message in the logs, so I would guess that it was on boot.
Given the location in the boot, I think it would be systemd subsystem.
Network interfaces would not start up and then none of the ceph services would start as there was no networking. (Ceph was unhealthy as is was missing the node and the OSD, MGR, and the MDS).
I would think that NIC would come up so as to get time from the NTP server before starting any ceph sesrvices.
No errors on boot screen that I could see. Just the error message in the logs, so I would guess that it was on boot.
Given the location in the boot, I think it would be systemd subsystem.
Network interfaces would not start up and then none of the ceph services would start as there was no networking. (Ceph was unhealthy as is was missing the node and the OSD, MGR, and the MDS).
I would think that NIC would come up so as to get time from the NTP server before starting any ceph sesrvices.
admin
2,918 Posts
December 31, 2021, 9:03 pmQuote from admin on December 31, 2021, 9:03 pmThis is not clear: after apt upgrade, you rebooted and the network interfaces would not start, yet after adjusting the time everything worked ?
This is not clear: after apt upgrade, you rebooted and the network interfaces would not start, yet after adjusting the time everything worked ?
podilarius
44 Posts
January 3, 2022, 4:13 amQuote from podilarius on January 3, 2022, 4:13 amI agree, but that is what seem to have happened. If I reboot a node, everything is working now.
After "apt upgrade" and reboot (many of them), the interface would not start unless I started them manually and then manually started all services.
After adjusting the time, all nodes restart without issues.
Will let you know after the next round of updates.
I agree, but that is what seem to have happened. If I reboot a node, everything is working now.
After "apt upgrade" and reboot (many of them), the interface would not start unless I started them manually and then manually started all services.
After adjusting the time, all nodes restart without issues.
Will let you know after the next round of updates.
Failed Dependency on setting up network
podilarius
44 Posts
Quote from podilarius on December 23, 2021, 6:02 pmAfter installation of a fresh install of 3.0.1 and cluster joins, I restarted node 1 for reboot test and network will not start unless I start it manually.
the python links are good, but on start up, I get:
syslog:Dec 23 17:48:12 prod-dbnk-cphsvr1 systemd[1]: Dependency failed for ifup for eth2.7.
syslog:Dec 23 17:48:12 prod-dbnk-cphsvr1 systemd[1]: ifup@eth2.7.service: Job ifup@eth2.7.service/start failed with result 'dependency'.If I complete this cycle on console, it all starts up.
systemctl restart networking
systemctl restart petasan*Let me know if I can provide anything.
After installation of a fresh install of 3.0.1 and cluster joins, I restarted node 1 for reboot test and network will not start unless I start it manually.
the python links are good, but on start up, I get:
syslog:Dec 23 17:48:12 prod-dbnk-cphsvr1 systemd[1]: Dependency failed for ifup for eth2.7.
syslog:Dec 23 17:48:12 prod-dbnk-cphsvr1 systemd[1]: ifup@eth2.7.service: Job ifup@eth2.7.service/start failed with result 'dependency'.
If I complete this cycle on console, it all starts up.
systemctl restart networking
systemctl restart petasan*
Let me know if I can provide anything.
admin
2,918 Posts
Quote from admin on December 23, 2021, 8:47 pm1) Double check your hardware/connections/ip configuration..etc
2) If still you have issue, first thing after reboot, please post:
ip addr list
/opt/petasan/config/cluster_info.json
/opt/petasan/config/node_info.json
/opt/petasan/log/PetaSAN.logif you are not able to ssh due to no ips at all, add ip via
ip addr add
instead of restarting any services, then scp the above
1) Double check your hardware/connections/ip configuration..etc
2) If still you have issue, first thing after reboot, please post:
ip addr list
/opt/petasan/config/cluster_info.json
/opt/petasan/config/node_info.json
/opt/petasan/log/PetaSAN.log
if you are not able to ssh due to no ips at all, add ip via
ip addr add
instead of restarting any services, then scp the above
podilarius
44 Posts
Quote from podilarius on December 23, 2021, 9:34 pm
- It was fine rebooting until I did ' apt update ; apt upgrade -y '.
- I have it up, I will send that when I get back to working on that system.
In the mean time, what is the best way to post file in here?
- It was fine rebooting until I did ' apt update ; apt upgrade -y '.
- I have it up, I will send that when I get back to working on that system.
In the mean time, what is the best way to post file in here?
admin
2,918 Posts
Quote from admin on December 23, 2021, 10:56 pmcan you also show
dpkg -l
you can share files on cloud storage (dropbox/google drive..) and post url
can you also show
dpkg -l
you can share files on cloud storage (dropbox/google drive..) and post url
podilarius
44 Posts
Quote from podilarius on December 28, 2021, 11:34 pmHere is a link to the files you are requesting. I will add the dpkg now.
https://drive.google.com/drive/folders/1UWsFAVKHo8ePqceDk38F_Es-47OElr6U?usp=sharing
Here is a link to the files you are requesting. I will add the dpkg now.
https://drive.google.com/drive/folders/1UWsFAVKHo8ePqceDk38F_Es-47OElr6U?usp=sharing
podilarius
44 Posts
Quote from podilarius on December 30, 2021, 4:52 pmCould this also be a function of unsynced time?
Since I corrected the clocks, everything seems to be working as advertised.
Could this also be a function of unsynced time?
Since I corrected the clocks, everything seems to be working as advertised.
admin
2,918 Posts
Quote from admin on December 31, 2021, 12:18 pmIt was fine rebooting until I did ' apt update ; apt upgrade -y '.
do you mean you got errors on your boot screen ? kernel errors / systemd errors ? network interfaces not up ? or was this just the Ceph cluster was not healthy ? if the later then yes the clock sync is important specially when you reboot multiple nodes at the same time, the cluster cannot get agreement on who has latest data, If the time sync among booting nodes does not correct itself quickly, make sure you hardware clock on your motherboards are all working correctly.
It was fine rebooting until I did ' apt update ; apt upgrade -y '.
do you mean you got errors on your boot screen ? kernel errors / systemd errors ? network interfaces not up ? or was this just the Ceph cluster was not healthy ? if the later then yes the clock sync is important specially when you reboot multiple nodes at the same time, the cluster cannot get agreement on who has latest data, If the time sync among booting nodes does not correct itself quickly, make sure you hardware clock on your motherboards are all working correctly.
podilarius
44 Posts
Quote from podilarius on December 31, 2021, 6:57 pmNo errors on boot screen that I could see. Just the error message in the logs, so I would guess that it was on boot.
Given the location in the boot, I think it would be systemd subsystem.Network interfaces would not start up and then none of the ceph services would start as there was no networking. (Ceph was unhealthy as is was missing the node and the OSD, MGR, and the MDS).
I would think that NIC would come up so as to get time from the NTP server before starting any ceph sesrvices.
No errors on boot screen that I could see. Just the error message in the logs, so I would guess that it was on boot.
Given the location in the boot, I think it would be systemd subsystem.
Network interfaces would not start up and then none of the ceph services would start as there was no networking. (Ceph was unhealthy as is was missing the node and the OSD, MGR, and the MDS).
I would think that NIC would come up so as to get time from the NTP server before starting any ceph sesrvices.
admin
2,918 Posts
Quote from admin on December 31, 2021, 9:03 pmThis is not clear: after apt upgrade, you rebooted and the network interfaces would not start, yet after adjusting the time everything worked ?
This is not clear: after apt upgrade, you rebooted and the network interfaces would not start, yet after adjusting the time everything worked ?
podilarius
44 Posts
Quote from podilarius on January 3, 2022, 4:13 amI agree, but that is what seem to have happened. If I reboot a node, everything is working now.
After "apt upgrade" and reboot (many of them), the interface would not start unless I started them manually and then manually started all services.
After adjusting the time, all nodes restart without issues.Will let you know after the next round of updates.
I agree, but that is what seem to have happened. If I reboot a node, everything is working now.
After "apt upgrade" and reboot (many of them), the interface would not start unless I started them manually and then manually started all services.
After adjusting the time, all nodes restart without issues.
Will let you know after the next round of updates.