Server not recognizing newly freed disk space
Clash Royale CLAN TAG#URR8PPP up vote
0
down vote
favorite
Ubuntu 14.04 on Amazon AWS EC2, has been in production for several years as a web server.
I was working on implementing a new backup process, and while doing so the server ran out of disk space. A 300gb magnetic drive, it's usage was at 290gb when I started this project. While backing up a rather large account, the drive ran out of space but was still mostly operational. OK, so I started deleting files. I deleted a lot of "temp" files local to a few particular web sites, I deleted some working backups (.gz files) that amounted to several gb, and I also checked the /tmp system folder and looked to see if there were any .Trashes folders. I believe in total I deleted at least 10gb of files. However my system still reported zero free space. And MySQL wouldn't start (which I know requires free space to start and operate).
I rebooted the server, to no avail. I shut down apache to make sure it wasn't holding open files. Eventually I expanded the disk drive to 350gb, and then everything started working. The server currently reports 298gb used. Should be much less than that, given all that I deleted.
1) Why would the server not recognize the space I was freeing up?
2) How can I get it to see it now?
Here is the output of df -h
Filesystem Size Used Avail Use% Mounted on
udev 1.8G 12K 1.8G 1% /dev
tmpfs 358M 200K 358M 1% /run
/dev/xvda1 345G 283G 47G 86% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
none 5.0M 0 5.0M 0% /run/lock
none 1.8G 0 1.8G 0% /run/shm
none 100M 0 100M 0% /run/user
overflow 1.0M 76K 948K 8% /tmp
14.04 disk-usage
asked May 15 at 17:14
Rob Brandt
1215
add a comment |Â
up vote
0
down vote
favorite
Ubuntu 14.04 on Amazon AWS EC2, has been in production for several years as a web server.
I was working on implementing a new backup process, and while doing so the server ran out of disk space. A 300gb magnetic drive, it's usage was at 290gb when I started this project. While backing up a rather large account, the drive ran out of space but was still mostly operational. OK, so I started deleting files. I deleted a lot of "temp" files local to a few particular web sites, I deleted some working backups (.gz files) that amounted to several gb, and I also checked the /tmp system folder and looked to see if there were any .Trashes folders. I believe in total I deleted at least 10gb of files. However my system still reported zero free space. And MySQL wouldn't start (which I know requires free space to start and operate).
I rebooted the server, to no avail. I shut down apache to make sure it wasn't holding open files. Eventually I expanded the disk drive to 350gb, and then everything started working. The server currently reports 298gb used. Should be much less than that, given all that I deleted.
1) Why would the server not recognize the space I was freeing up?
2) How can I get it to see it now?
Here is the output of df -h
Filesystem Size Used Avail Use% Mounted on
udev 1.8G 12K 1.8G 1% /dev
tmpfs 358M 200K 358M 1% /run
/dev/xvda1 345G 283G 47G 86% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
none 5.0M 0 5.0M 0% /run/lock
none 1.8G 0 1.8G 0% /run/shm
none 100M 0 100M 0% /run/user
overflow 1.0M 76K 948K 8% /tmp
14.04 disk-usage
asked May 15 at 17:14
Rob Brandt
1215
And while it seemed to be working when I wrote the above, I just now see that I cannot create a cron job because of lack of space.
– Rob Brandt
May 15 at 17:23
Can you edit your question to include the output ofdf -hplease?
– muclux
May 15 at 17:57
After changing the size of the physical disk, did you runresize2fsto resize the filesystem? Readman resize2fs.
– waltinator
May 16 at 4:14
@waltinator, this is an issue that is confusing to me. I have been using Ubuntu on AWS since the 10.04 days, and until I installed a 16.04 server I have NEVER had to resize the file system. It doesn't appear that I need to now either. When I run lsblk -o NAME,FSTYPE,SIZE,MOUNTPOINT,LABEL it shows just one entry, at 350gb. When I run fdisk -l, I get similar data, ending with "Disk /dev/xvda1 doesn't contain a valid partition table". So I guess since my disk isn't split between partitions, I don't have to resize(?). So I guess the reason I don't need to resize is that my disk i
– Rob Brandt
May 16 at 21:02
I believe i have solved the "out of disk space" issue from here askubuntu.com/questions/199565/not-enough-space-on-tmp using option 2 from the approved answer. You can see from my table above that /tmp is mapped to overflow, which apparently is a stopgap Ubuntu does when it runs out of disk space. And then I ran out of overflow space. The solution was to clean out more disk space (down to 280gb used) and reboot. overflow is now gone and it appears /tmp is using regular disk space again.
– Rob Brandt
May 16 at 21:06
add a comment |Â
up vote
0
down vote
favorite
up vote
0
down vote
favorite
Ubuntu 14.04 on Amazon AWS EC2, has been in production for several years as a web server.
I was working on implementing a new backup process, and while doing so the server ran out of disk space. A 300gb magnetic drive, it's usage was at 290gb when I started this project. While backing up a rather large account, the drive ran out of space but was still mostly operational. OK, so I started deleting files. I deleted a lot of "temp" files local to a few particular web sites, I deleted some working backups (.gz files) that amounted to several gb, and I also checked the /tmp system folder and looked to see if there were any .Trashes folders. I believe in total I deleted at least 10gb of files. However my system still reported zero free space. And MySQL wouldn't start (which I know requires free space to start and operate).
I rebooted the server, to no avail. I shut down apache to make sure it wasn't holding open files. Eventually I expanded the disk drive to 350gb, and then everything started working. The server currently reports 298gb used. Should be much less than that, given all that I deleted.
1) Why would the server not recognize the space I was freeing up?
2) How can I get it to see it now?
Here is the output of df -h
Filesystem Size Used Avail Use% Mounted on
udev 1.8G 12K 1.8G 1% /dev
tmpfs 358M 200K 358M 1% /run
/dev/xvda1 345G 283G 47G 86% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
none 5.0M 0 5.0M 0% /run/lock
none 1.8G 0 1.8G 0% /run/shm
none 100M 0 100M 0% /run/user
overflow 1.0M 76K 948K 8% /tmp
14.04 disk-usage
asked May 15 at 17:14
Rob Brandt
1215
Ubuntu 14.04 on Amazon AWS EC2, has been in production for several years as a web server.
I was working on implementing a new backup process, and while doing so the server ran out of disk space. A 300gb magnetic drive, it's usage was at 290gb when I started this project. While backing up a rather large account, the drive ran out of space but was still mostly operational. OK, so I started deleting files. I deleted a lot of "temp" files local to a few particular web sites, I deleted some working backups (.gz files) that amounted to several gb, and I also checked the /tmp system folder and looked to see if there were any .Trashes folders. I believe in total I deleted at least 10gb of files. However my system still reported zero free space. And MySQL wouldn't start (which I know requires free space to start and operate).
I rebooted the server, to no avail. I shut down apache to make sure it wasn't holding open files. Eventually I expanded the disk drive to 350gb, and then everything started working. The server currently reports 298gb used. Should be much less than that, given all that I deleted.
1) Why would the server not recognize the space I was freeing up?
2) How can I get it to see it now?
Here is the output of df -h
Filesystem Size Used Avail Use% Mounted on
udev 1.8G 12K 1.8G 1% /dev
tmpfs 358M 200K 358M 1% /run
/dev/xvda1 345G 283G 47G 86% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
none 5.0M 0 5.0M 0% /run/lock
none 1.8G 0 1.8G 0% /run/shm
none 100M 0 100M 0% /run/user
overflow 1.0M 76K 948K 8% /tmp
14.04 disk-usage
asked May 15 at 17:14
Rob Brandt
1215
edited May 15 at 22:24
asked May 15 at 17:14
Rob Brandt
1215
asked May 15 at 17:14
Rob Brandt
1215
asked May 15 at 17:14
Rob Brandt
1215
1215
And while it seemed to be working when I wrote the above, I just now see that I cannot create a cron job because of lack of space.
– Rob Brandt
May 15 at 17:23
Can you edit your question to include the output ofdf -hplease?
– muclux
May 15 at 17:57
After changing the size of the physical disk, did you runresize2fsto resize the filesystem? Readman resize2fs.
– waltinator
May 16 at 4:14
@waltinator, this is an issue that is confusing to me. I have been using Ubuntu on AWS since the 10.04 days, and until I installed a 16.04 server I have NEVER had to resize the file system. It doesn't appear that I need to now either. When I run lsblk -o NAME,FSTYPE,SIZE,MOUNTPOINT,LABEL it shows just one entry, at 350gb. When I run fdisk -l, I get similar data, ending with "Disk /dev/xvda1 doesn't contain a valid partition table". So I guess since my disk isn't split between partitions, I don't have to resize(?). So I guess the reason I don't need to resize is that my disk i
– Rob Brandt
May 16 at 21:02
I believe i have solved the "out of disk space" issue from here askubuntu.com/questions/199565/not-enough-space-on-tmp using option 2 from the approved answer. You can see from my table above that /tmp is mapped to overflow, which apparently is a stopgap Ubuntu does when it runs out of disk space. And then I ran out of overflow space. The solution was to clean out more disk space (down to 280gb used) and reboot. overflow is now gone and it appears /tmp is using regular disk space again.
– Rob Brandt
May 16 at 21:06
add a comment |Â
And while it seemed to be working when I wrote the above, I just now see that I cannot create a cron job because of lack of space.
– Rob Brandt
May 15 at 17:23
Can you edit your question to include the output ofdf -hplease?
– muclux
May 15 at 17:57
After changing the size of the physical disk, did you runresize2fsto resize the filesystem? Readman resize2fs.
– waltinator
May 16 at 4:14
@waltinator, this is an issue that is confusing to me. I have been using Ubuntu on AWS since the 10.04 days, and until I installed a 16.04 server I have NEVER had to resize the file system. It doesn't appear that I need to now either. When I run lsblk -o NAME,FSTYPE,SIZE,MOUNTPOINT,LABEL it shows just one entry, at 350gb. When I run fdisk -l, I get similar data, ending with "Disk /dev/xvda1 doesn't contain a valid partition table". So I guess since my disk isn't split between partitions, I don't have to resize(?). So I guess the reason I don't need to resize is that my disk i
– Rob Brandt
May 16 at 21:02
I believe i have solved the "out of disk space" issue from here askubuntu.com/questions/199565/not-enough-space-on-tmp using option 2 from the approved answer. You can see from my table above that /tmp is mapped to overflow, which apparently is a stopgap Ubuntu does when it runs out of disk space. And then I ran out of overflow space. The solution was to clean out more disk space (down to 280gb used) and reboot. overflow is now gone and it appears /tmp is using regular disk space again.
– Rob Brandt
May 16 at 21:06
And while it seemed to be working when I wrote the above, I just now see that I cannot create a cron job because of lack of space.
– Rob Brandt
May 15 at 17:23
And while it seemed to be working when I wrote the above, I just now see that I cannot create a cron job because of lack of space.
– Rob Brandt
May 15 at 17:23
Can you edit your question to include the output of
df -h please?– muclux
May 15 at 17:57
Can you edit your question to include the output of
df -h please?– muclux
May 15 at 17:57
After changing the size of the physical disk, did you run
resize2fs to resize the filesystem? Read man resize2fs.– waltinator
May 16 at 4:14
After changing the size of the physical disk, did you run
resize2fs to resize the filesystem? Read man resize2fs.– waltinator
May 16 at 4:14
@waltinator, this is an issue that is confusing to me. I have been using Ubuntu on AWS since the 10.04 days, and until I installed a 16.04 server I have NEVER had to resize the file system. It doesn't appear that I need to now either. When I run lsblk -o NAME,FSTYPE,SIZE,MOUNTPOINT,LABEL it shows just one entry, at 350gb. When I run fdisk -l, I get similar data, ending with "Disk /dev/xvda1 doesn't contain a valid partition table". So I guess since my disk isn't split between partitions, I don't have to resize(?). So I guess the reason I don't need to resize is that my disk i
– Rob Brandt
May 16 at 21:02
@waltinator, this is an issue that is confusing to me. I have been using Ubuntu on AWS since the 10.04 days, and until I installed a 16.04 server I have NEVER had to resize the file system. It doesn't appear that I need to now either. When I run lsblk -o NAME,FSTYPE,SIZE,MOUNTPOINT,LABEL it shows just one entry, at 350gb. When I run fdisk -l, I get similar data, ending with "Disk /dev/xvda1 doesn't contain a valid partition table". So I guess since my disk isn't split between partitions, I don't have to resize(?). So I guess the reason I don't need to resize is that my disk i
– Rob Brandt
May 16 at 21:02
I believe i have solved the "out of disk space" issue from here askubuntu.com/questions/199565/not-enough-space-on-tmp using option 2 from the approved answer. You can see from my table above that /tmp is mapped to overflow, which apparently is a stopgap Ubuntu does when it runs out of disk space. And then I ran out of overflow space. The solution was to clean out more disk space (down to 280gb used) and reboot. overflow is now gone and it appears /tmp is using regular disk space again.
– Rob Brandt
May 16 at 21:06
I believe i have solved the "out of disk space" issue from here askubuntu.com/questions/199565/not-enough-space-on-tmp using option 2 from the approved answer. You can see from my table above that /tmp is mapped to overflow, which apparently is a stopgap Ubuntu does when it runs out of disk space. And then I ran out of overflow space. The solution was to clean out more disk space (down to 280gb used) and reboot. overflow is now gone and it appears /tmp is using regular disk space again.
– Rob Brandt
May 16 at 21:06
add a comment |Â
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
var $window = $(window),
onScroll = function(e)
var $elem = $('.new-login-left'),
docViewTop = $window.scrollTop(),
docViewBottom = docViewTop + $window.height(),
elemTop = $elem.offset().top,
elemBottom = elemTop + $elem.height();
if ((docViewTop elemBottom))
StackExchange.using('gps', function() StackExchange.gps.track('embedded_signup_form.view', location: 'question_page' ); );
$window.unbind('scroll', onScroll);
;
$window.on('scroll', onScroll);
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f1036626%2fserver-not-recognizing-newly-freed-disk-space%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
var $window = $(window),
onScroll = function(e)
var $elem = $('.new-login-left'),
docViewTop = $window.scrollTop(),
docViewBottom = docViewTop + $window.height(),
elemTop = $elem.offset().top,
elemBottom = elemTop + $elem.height();
if ((docViewTop elemBottom))
StackExchange.using('gps', function() StackExchange.gps.track('embedded_signup_form.view', location: 'question_page' ); );
$window.unbind('scroll', onScroll);
;
$window.on('scroll', onScroll);
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
var $window = $(window),
onScroll = function(e)
var $elem = $('.new-login-left'),
docViewTop = $window.scrollTop(),
docViewBottom = docViewTop + $window.height(),
elemTop = $elem.offset().top,
elemBottom = elemTop + $elem.height();
if ((docViewTop elemBottom))
StackExchange.using('gps', function() StackExchange.gps.track('embedded_signup_form.view', location: 'question_page' ); );
$window.unbind('scroll', onScroll);
;
$window.on('scroll', onScroll);
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
var $window = $(window),
onScroll = function(e)
var $elem = $('.new-login-left'),
docViewTop = $window.scrollTop(),
docViewBottom = docViewTop + $window.height(),
elemTop = $elem.offset().top,
elemBottom = elemTop + $elem.height();
if ((docViewTop elemBottom))
StackExchange.using('gps', function() StackExchange.gps.track('embedded_signup_form.view', location: 'question_page' ); );
$window.unbind('scroll', onScroll);
;
$window.on('scroll', onScroll);
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
And while it seemed to be working when I wrote the above, I just now see that I cannot create a cron job because of lack of space.
– Rob Brandt
May 15 at 17:23
Can you edit your question to include the output of
df -hplease?– muclux
May 15 at 17:57
After changing the size of the physical disk, did you run
resize2fsto resize the filesystem? Readman resize2fs.– waltinator
May 16 at 4:14
@waltinator, this is an issue that is confusing to me. I have been using Ubuntu on AWS since the 10.04 days, and until I installed a 16.04 server I have NEVER had to resize the file system. It doesn't appear that I need to now either. When I run lsblk -o NAME,FSTYPE,SIZE,MOUNTPOINT,LABEL it shows just one entry, at 350gb. When I run fdisk -l, I get similar data, ending with "Disk /dev/xvda1 doesn't contain a valid partition table". So I guess since my disk isn't split between partitions, I don't have to resize(?). So I guess the reason I don't need to resize is that my disk i
– Rob Brandt
May 16 at 21:02
I believe i have solved the "out of disk space" issue from here askubuntu.com/questions/199565/not-enough-space-on-tmp using option 2 from the approved answer. You can see from my table above that /tmp is mapped to overflow, which apparently is a stopgap Ubuntu does when it runs out of disk space. And then I ran out of overflow space. The solution was to clean out more disk space (down to 280gb used) and reboot. overflow is now gone and it appears /tmp is using regular disk space again.
– Rob Brandt
May 16 at 21:06