Apache – No space left on device / Failed to create proxy Mutex
Posted by Sriram Sanka on August 10, 2018
A Big Thanks to “https://help.directadmin.com/item.php?id=110”
Restarting Apache in one of the Servers failed with “No Space left on the Device ” Error and we have enough space in all available partitions.
This is something to deal with Server Kernel semaphores.
“On Linux, A semaphore is a System V IPC object that is used to control utilization of a particular process. Refer https://www.tldp.org/LDP/tlk/ipc/ipc.html
Semaphores are a shareable resource that take on a non-negative integer value. They are manipulated by the P (wait) and V (signal) functions, which decrement and increment the semaphore, respectively. When a process needs a resource, a “wait” is issued and the semaphore is decremented. When the semaphore contains a value of zero, the resources are not available and the calling process spins or blocks (as appropriate) until resources are available. When a process releases a resource controlled by a semaphore, it increments the semaphore and the waiting processes are notified.”
The system does n`t have actual resource available to serve the request, so either we need to configure the Kernel Semaphores or clear the Old entries to get it back.
kernel.sem = SEMMSL SEMMNS SEMOPM SEMMNI
SEMMSL maximum number of semaphores per array
SEMMNS maximum semaphores system-wide
SEMOPM maximum operations per semop call
SEMMNI maximum arrays
[root@r12 ~]# cat /etc/sysctl.conf |grep kernel.sem
kernel.sem = 256 32000 100 142
[root@r12 ~]#[root@r12 ~]# cat /proc/sys/kernel/sem
256 32000 100 142
[root@r12 ~]#
[root@r12 ~]# ipcs -l | awk ‘FNR>=7 && FNR<=15’
—— Semaphore Limits ——–
max number of arrays = 142
max semaphores per array = 256
max semaphores system wide = 32000
max ops per semop call = 100
semaphore max value = 32767—— Messages: Limits ——–
[root@r12 ~]#[root@r12 ~]# ipcs -ls
—— Semaphore Limits ——–
max number of arrays = 142
max semaphores per array = 256
max semaphores system wide = 32000
max ops per semop call = 100
semaphore max value = 32767
To Clear , we can execute the following
ipcs | grep apache | awk ‘{print $2}’ > sem.txt
for i in `cat sem.txt`; do { ipcrm -s $i; }; done;
For Automating the removal of apache semaphores with ipcs/ipcrm, Please refer https://help.directadmin.com/item.php?id=572 |
Leave a Reply