Physical content behind system-related mountpoints

Here is the place were the team and the community projects meet together. Help us to develop SolydXK projects or contribute your ideas for future releases.
User avatar
Arjen Balfoort
Site Admin
Posts: 9258
Joined: 26 Jan 2013 19:36
Location: Netherlands
Contact:

Re: Physical content behind system-related mountpoints

Postby Arjen Balfoort » 25 Jun 2015 12:22

I never saw the lsblk message. I know of the "device node not found" message but ignored it because I couldn't find the cause and the Grub was created just the same.


SolydXK needs you!
Development | Testing | Translations

User avatar
grizzler
Posts: 2160
Joined: 04 Mar 2013 15:45
Location: The Hague, NL

Re: Physical content behind system-related mountpoints

Postby grizzler » 25 Jun 2015 12:37

There's actually supposed to be a 'Found linux image:' and 'Found initrd image:' line in there as well (missed them, because there are an awful lot of 'device node not found' lines in the logged output...).
Anyway, this is what happens when /sys isn't mounted. As you found, it does no real harm, because it's only about the grub.cfg file in the ISO's root path which is still created (and never used anyway).
Frank

SolydX EE 64 - tracking Debian Testing

User avatar
Arjen Balfoort
Site Admin
Posts: 9258
Joined: 26 Jan 2013 19:36
Location: Netherlands
Contact:

Re: Physical content behind system-related mountpoints

Postby Arjen Balfoort » 25 Jun 2015 13:25

I am now experimenting with these mounts:

Code: Select all

            self.ec.run("mount --bind /proc '%s'" % proc)
            self.ec.run("mount --bind /dev '%s'" % dev)
            self.ec.run("mount --bind /sys '%s'" % sys)
            self.ec.run("mount --bind /run '%s'" % run)
(leaving out /dev/pts)

Seems to work without errors with update-grub and doing an upgrade in solydxk-constructor.
Don't know if it's wise to bind /sys and /run in a chrooted environment, though.


SolydXK needs you!
Development | Testing | Translations

User avatar
grizzler
Posts: 2160
Joined: 04 Mar 2013 15:45
Location: The Hague, NL

Re: Physical content behind system-related mountpoints

Postby grizzler » 25 Jun 2015 13:58

Binding /sys appears to be standard. It's done on nearly every site I checked. I'm not sure about /run. As I wrote, there are a few sites where /run is included (1, 2, 3 - I've actually found two more - 4, 5 - yes, I know, these aren't about building ISOs...), but others don't even mention its existence.

It's curious that nobody seems to have written the 'definitive guide on chrooting'.
Frank

SolydX EE 64 - tracking Debian Testing

User avatar
Arjen Balfoort
Site Admin
Posts: 9258
Joined: 26 Jan 2013 19:36
Location: Netherlands
Contact:

Re: Physical content behind system-related mountpoints

Postby Arjen Balfoort » 25 Jun 2015 14:26

Indeed, since I've started SolydXK it was mostly trial and error when building the ISOs.

I also just found out you need to mount /dev/pts as well when building the ISOs.


SolydXK needs you!
Development | Testing | Translations

User avatar
grizzler
Posts: 2160
Joined: 04 Mar 2013 15:45
Location: The Hague, NL

Re: Physical content behind system-related mountpoints

Postby grizzler » 25 Jun 2015 14:35

LOL! I was just about to ask you whether leaving out /dev/pts didn't produce errors like the one mentioned here (Can not write log, openpty() failed (/dev/pts not mounted?)).
Frank

SolydX EE 64 - tracking Debian Testing

User avatar
Arjen Balfoort
Site Admin
Posts: 9258
Joined: 26 Jan 2013 19:36
Location: Netherlands
Contact:

Re: Physical content behind system-related mountpoints

Postby Arjen Balfoort » 25 Jun 2015 15:06

Mounting /run in chroot crashes the host :mrgreen:
That now leaves me with an error on a missing /run/lock


SolydXK needs you!
Development | Testing | Translations

User avatar
grizzler
Posts: 2160
Joined: 04 Mar 2013 15:45
Location: The Hague, NL

Re: Physical content behind system-related mountpoints

Postby grizzler » 25 Jun 2015 15:35

I was afraid something like that might happen...

Creating a /run/lock directory inside the chrooted environment it is, then.

Edit
Seems to have the same set of permissions as /tmp: 1777.
Frank

SolydX EE 64 - tracking Debian Testing

User avatar
grizzler
Posts: 2160
Joined: 04 Mar 2013 15:45
Location: The Hague, NL

Re: Physical content behind system-related mountpoints

Postby grizzler » 25 Jun 2015 16:43

A test update/upgrade on one of the ISOs with my script (more or less what the Constructor does when it runs cleanup.sh) ran correctly, except for some strangeness to do with the boot-isos.sh script:

Code: Select all

Generating grub configuration file ...
Found linux image: /boot/vmlinuz-3.16.0-4-amd64
Found initrd image: /boot/initrd.img-3.16.0-4-amd64
Mount is denied because the NTFS volume is already exclusively opened.
The volume may be already mounted, or another software may use it which
could be identified for example by the help of the 'fuser' command.
rmdir: failed to remove '/mnt/b5': Device or resource busy
rmdir: failed to remove '/mnt/b7': Device or resource busy
rmdir: failed to remove '/mnt/b8': Device or resource busy
Found SolydX EE 32-bit (1) on /dev/sdb5
Found SolydX 64-bit (1) on /dev/sdb6
Found SolydK 64-bit (1) on /dev/sdb7
Found LMDE 2 Betsy (2) on /dev/sdb8
It seems boot-isos.sh tried to look at one of my data partitions (the NTFS formatted one) and failed. Also, it couldn't remove 3 of the 4 temporary mount directories for sdb from the build structure's /mnt, apparently because unmounting the partitions from those directories didn't work (they were indeed still mounted there afterwards), which is odd. Why did it fail with sdb5, 7 and 8 but not with 6?

Looking at this, I think bind mounting /sys may be more trouble than it's worth (at least for me). I'll probably remove it again, unless I can find some way to block boot-isos.sh from running. It's not as if that's doing anything useful while a build structure is being updated anyway.

Temporarily creating /run/lock worked like a charm, by the way.

Edit
I just noticed I'll have to make it remove /run/blkid as well... ;)
Frank

SolydX EE 64 - tracking Debian Testing

User avatar
Arjen Balfoort
Site Admin
Posts: 9258
Joined: 26 Jan 2013 19:36
Location: Netherlands
Contact:

Re: Physical content behind system-related mountpoints

Postby Arjen Balfoort » 25 Jun 2015 18:30

I've updated solydxk-constructor and uploading nightlies now.


SolydXK needs you!
Development | Testing | Translations

User avatar
bas_otten
Posts: 212
Joined: 19 Oct 2013 12:22
Location: Netherlands

Re: Physical content behind system-related mountpoints

Postby bas_otten » 25 Jun 2015 19:53

grizzler wrote: I just noticed I'll have to make it remove /run/blkid as well.
Well, the /run/blkid is (also) created during installation of the ISO by the user, not (just) while building the ISO. So if I'm with you on this, running your cleanup script right before creating the ISO will not make a difference for an installed system. In that case I think the Installer should handle this.

Thanks Arjen and Frank, for picking up this issue so adequately. I had never guessed there would be such a story behind this, related to how the ISO is built in the first place!

Since we're in to cleaning the house now anyway, I discovered one more location of obsolete files: /var/backups. A view from SystemRescueCD is:

Code: Select all

root@sysresccd /mnt/var % l backups
total 6676
655397 drwxr-xr-x  2 root root    4096 Jul 28  2012 .
655361 drwxr-xr-x 11 root root    4096 Feb 20  2013 ..
655437 -rw-r--r--  1 root root   29064 Jul 28  2012 apt.extended_states.0
655441 -rw-r--r--  1 root root    4521 Sep  8  2011 apt.extended_states.1.gz
655440 -rw-r--r--  1 root root    4276 Jul 24  2011 apt.extended_states.2.gz
655401 -rw-r--r--  1 root root 2933746 Jul 24  2012 aptitude.pkgstates.0
655403 -rw-r--r--  1 root root  264231 Sep  8  2011 aptitude.pkgstates.1.gz
655404 -rw-r--r--  1 root root  262095 Jul 24  2011 aptitude.pkgstates.2.gz
655399 -rw-r--r--  1 root root 1644770 Jul 28  2012 dpkg.status.0
655400 -rw-r--r--  1 root root  410620 Mar 26  2012 dpkg.status.1.gz
655405 -rw-r--r--  1 root root  410642 Mar 24  2012 dpkg.status.2.gz
655402 -rw-r--r--  1 root root  413618 Mar 24  2012 dpkg.status.3.gz
655398 -rw-r--r--  1 root root  407895 Jul 24  2011 dpkg.status.4.gz
655436 -rw-------  1 root root     908 Jul 24  2012 group.bak
655438 -rw-------  1 root   42     754 Jul 24  2012 gshadow.bak
655442 -rw-------  1 root root    1638 Jul 24  2012 passwd.bak
655439 -rw-------  1 root   42     952 Jul 24  2012 shadow.bak
root@sysresccd /mnt/var % du -ks backups
6672    backups
It is close to 7MB of 2011/2012 backups.

But in contrast to the previously discussed directories, this is not a mountpoint, so technically probably an easy one to clean up script-wise. I took a tour through the whole filesystem, think this will be my final suggestion in this area :)

User avatar
grizzler
Posts: 2160
Joined: 04 Mar 2013 15:45
Location: The Hague, NL

Re: Physical content behind system-related mountpoints

Postby grizzler » 25 Jun 2015 19:58

From /usr/lib/solydxk/constructor/cleanup.sh:

Code: Select all

rm -rf /media/*
rm -rf /mnt/*
Are you really sure you want to do that? If I had done that after I noticed those three /mnt subdirectories were still there, I would have wiped three complete installs. This is mounting, not symlinking.
Frank

SolydX EE 64 - tracking Debian Testing

User avatar
grizzler
Posts: 2160
Joined: 04 Mar 2013 15:45
Location: The Hague, NL

Re: Physical content behind system-related mountpoints

Postby grizzler » 25 Jun 2015 20:09

bas_otten wrote:Since we're in to cleaning the house now anyway, I discovered one more location of obsolete files: /var/backups.
Ah yes, I mentioned that in one of my postings in the 'ISO cleanup' thread in the Developers & Maintainers subforum (http://forums.solydxk.nl/viewtopic.php? ... 325#p52420 - only accessible to the team, I'm afraid). I didn't realise Arjen hadn't picked it up. It's already been handled in the CE ISOs.
Frank

SolydX EE 64 - tracking Debian Testing

User avatar
Arjen Balfoort
Site Admin
Posts: 9258
Joined: 26 Jan 2013 19:36
Location: Netherlands
Contact:

Re: Physical content behind system-related mountpoints

Postby Arjen Balfoort » 25 Jun 2015 21:27

Must have missed it. Well, I've updated the constructor again (although I think I'm the only one using it).

B.t.w., is /srv used for anything? It always seems empty.


SolydXK needs you!
Development | Testing | Translations

User avatar
grizzler
Posts: 2160
Joined: 04 Mar 2013 15:45
Location: The Hague, NL

Re: Physical content behind system-related mountpoints

Postby grizzler » 26 Jun 2015 06:24

The Linux Foundation's Filesystem Hierarchy Standard has this to say about it:
3.17. /srv : Data for services provided by this system

3.17.1. Purpose

/srv contains site-specific data which is served by this system.

Rationale

This main purpose of specifying this is so that users may find the location of the data files for a particular service, and so that services which require a single tree for readonly data, writable data and scripts (such as cgi scripts) can be reasonably placed. Data that is only of interest to a specific user should go in that users' home directory. If the directory and file structure of the data is not exposed to consumers, it should go in /var/lib.

The methodology used to name subdirectories of /srv is unspecified as there is currently no consensus on how this should be done. One method for structuring data under /srv is by protocol, eg. ftp, rsync, www, and cvs. On large systems it can be useful to structure /srv by administrative context, such as /srv/physics/www, /srv/compsci/cvs, etc. This setup will differ from host to host. Therefore, no program should rely on a specific subdirectory structure of /srv existing or data necessarily being stored in /srv. However /srv should always exist on FHS compliant systems and should be used as the default location for such data.

Distributions must take care not to remove locally placed files in these directories without administrator permission.
So in an ISO, it should be empty. On my main machine it holds my local repository and the local copies of the websites I maintain.
Frank

SolydX EE 64 - tracking Debian Testing

User avatar
Arjen Balfoort
Site Admin
Posts: 9258
Joined: 26 Jan 2013 19:36
Location: Netherlands
Contact:

Re: Physical content behind system-related mountpoints

Postby Arjen Balfoort » 26 Jun 2015 06:28

Thanks. Learned something today and it's still early.


SolydXK needs you!
Development | Testing | Translations

User avatar
grizzler
Posts: 2160
Joined: 04 Mar 2013 15:45
Location: The Hague, NL

Re: Physical content behind system-related mountpoints

Postby grizzler » 26 Jun 2015 17:05

Schoelje wrote:...uploading nightlies now.
Did something go wrong there? They don't seem to have made it to the download page.
Frank

SolydX EE 64 - tracking Debian Testing

User avatar
Arjen Balfoort
Site Admin
Posts: 9258
Joined: 26 Jan 2013 19:36
Location: Netherlands
Contact:

Re: Physical content behind system-related mountpoints

Postby Arjen Balfoort » 26 Jun 2015 17:24

Real life caught up. Will do later.


SolydXK needs you!
Development | Testing | Translations


Return to “Suggestions & ideas / Open Projects”

Who is online

Users browsing this forum: No registered users and 1 guest