coupleprogramming

So, this is the last progress report of this years GSoC. That doesn’t mean that this project is done, but GSoC has come to an end.

To wrap things up, the version, which will hopefully be added to the tree soon is the one which sets the caps directly in the livefs. Why? Because I still haven’t figured out a good fallback-mechanism, if the livefs doesn’t support caps and the caps can’t be reapplied after the copy.
Don’t worry, I’m working on that, but it won’t be finished by Aug 16th, which is the Pencils Down Date of GSoC. So flameeyes and me decided, that it would be good, to see this version as a final result.

Thanks you all for trying my stuff and commenting. This was the best summer of my life so far, I learned a lot and had a great time. Thanks to my mentor Diego E. “Flameeyes” Pettenò for giving me feedback and encouraging me, when I wasn’t so sure I’m on the right path.

As for my involvement with Gentoo, there is already a recruitment bug for me in bugzilla ;).

Again, it has been two weeks since my last report, shame on me. But my exams are all written by now (and passed :D) and I have some exciting new stuff to tell you :).

What did I do this week (and the last):
The next step in this project was to find out why and where the caps get lost, when they are set in src_install (that means on the binary in the sandbox and not on the binary which is already in the livefs).
After a long and sometimes frustrating journey through the portage sources and with a lot help from pdb I found out why.
Normally (e.g. when the source and the destination are on the same filesystem) os.rename is used to move the binary from sandbox to livefs. This happens in portage/pym/portage/util/movefile.py and is not were the caps are lost.
They are lost earlier, when the binaries are stripped. So I added a preserving-mechanism to portage/bin/ebuild-helpers/prepstrip, where the stripping happens. And it worked :). I could set the caps in src_install and they did not get lost.
But there was another place where the caps could get lost. If os.rename in movefile.py fails or src and dest are not on the same filesystem, shutil.copy is used before os.rename and the binarys mode/ownership are restored after. Of course the extended attributes, like caps, have to be restored, too. Using pyxattr I extracted the caps before the copy and applied them back after the binary is copied and its mode/ownership are restored. The downside is, that portage now needs pyxattr.
With that now working, I changed all ebuilds I have patched so far, so the caps are set in src_install.
I added the changed ebuilds and the changed portage-files to a new branch in my repository. I used the current stable version (2.1.8.3) of portage, just so you know :).

That’s it for today, tell me what you think in the comments :).

This report is a little late, but I’m in my exams period and had to do a lot of studying. I had three exams already and have two to go next week, so I will be back full time, after that. This report is about this and the last week, enjoy :D.

What did I do this week (and the last):
Since some “easy” ebuilds have been patched, flameeyes suggested, I could take a look at tcpdump. While tcpdump has no suid-bit set, it can only be run as root, which is inconvenient. It would be nice if “special” users could run it to sniff (without using sudo) and normal users can use it to view dumps. Then it could also be moved to /usr/bin/. So how could you realize that with caps?
There is a pam module called pam_cap, which allows you to put caps in the users inherited set.
Then there are two options:

1. Is your application capability-aware?
   You can put the caps in the permitted set and/or inherited set.
2. Is your application not capability-aware?
   You will have to put the caps in the inherited and effective set.

Only capability-aware application can populate their effective set themselves, the kernel has to do it for the others. And the kernel knows, if the effective set is not empty, the application is not capability-aware [0].

So, how do you use pam_cap?

1. You need to have libcap installed. (Obviously :))
2. You need to include pam_cap.so in your /etc/pam.d/system-auth.
   Something like this:
   auth required pam_env.so
auth required pam_cap.so
...
3. Add caps to users in /etc/security/capability.conf.
   Like this:
   #netadmin inherits cap_net_admin,cap_net_raw
cap_net_admin,cap_net_raw netadmin
#every other user inherits nothing
none *
   The last line is very important, otherwise all other users inherit all caps, which would be not what we wanted.
4. Logout/Login
5. It should work now, but maybe you need to reboot.

Now, that you can grant users certain caps, you can put the caps on your binaries in ei or p/pi instead of ep and only the users, who have the right caps in their set can execute these binaries correctly. To try it out you can emerge tcpdump from my overlay, which has cap_net_raw in its effective and inherited set. Only a user who has at least cap_net_raw in his/her set can sniff with tcpdump.

You want to see this as an example, before you try it out on your system? Here you go:
This is my /etc/pam.d/system-auth:

auth required pam_env.so
auth required pam_cap.so
auth required pam_unix.so try_first_pass likeauth nullok

account required pam_unix.so

password required pam_cracklib.so difok=2 minlen=8 dcredit=2 ocredit=2 retry=3
password required pam_unix.so try_first_pass use_authtok nullok sha512 shadow
session required pam_limits.so
session required pam_env.so
session required pam_unix.so
session optional pam_permit.so

This is my /etc/security/capabiliy.conf

#
# /etc/security/capability.conf
#
# this is a sample capability file (to be used in conjunction with
# the pam_cap.so module)
#
# In order to use this module, it must have been linked with libcap
# and thus you’ll know about Linux’s capability support.
# [If you don't know about libcap, the sources for it are here:
#
# http://linux.kernel.org/pub/linux/libs/security/linux-privs/
#
# .]
#
# Here are some sample lines (remove the preceding ‘#’ if you want to
# use them

## user ‘morgan’ gets the CAP_SETFCAP inheritable capability
##cap_setfcap morgan

cap_net_admin,cap_net_raw netadmin

## ‘everyone else’ gets no inheritable capabilities
none *

## if there is no ‘*’ entry, all users not explicitly mentioned will
## get all available capabilities. This is a permissive default, and
## probably not what you want…

tcpdump is now in /usr/bin/, let’s look at its caps.
superkfr@totoro ~ $ sudo getcap /usr/bin/tcpdump
/usr/bin/tcpdump = cap_net_raw+ei
It has cap_net_raw in the effective and inherited set, good so far.

Let’s try to execute tcpdump with the user netadmin, who has the required caps.
superkfr@totoro ~ $ su - netadmin
Password:
netadmin@totoro ~ $ tcpdump -i wlan0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on wlan0, link-type EN10MB (Ethernet), capture size 65535 bytes
...
netadmin@totoro ~ $ exit
logout
Works just fine.

Let’s try with user test, who has no caps inherited.
superkfr@totoro ~ $ su - test
Password:
test@totoro ~ $ tcpdump -i wlan0
tcpdump: wlan0: You don't have permission to capture on that device
(socket: Operation not permitted)
test@totoro ~ $ tcpdump -r /tmp/dump
reading from file /tmp/dump, link-type EN10MB (Ethernet)
...
test@totoro ~ $ exit
logout

Like we expected, user test does not have the right permissions to sniff, but can view dumps.

If you have trouble getting pam_cap to work (believe me, I’ve been there), try this:
Look at the output of /sbin/getpcaps $$ in the shell of the user, whom you want to have the caps.

• If the output looks something like this:
  test@totoro ~ $ /sbin/getpcaps $$
Capabilities for `8219': =
  The users cap set is empty and something didn’t work. Try rebooting, if you haven’t done this already.
• If the output looks something like this:
  netadmin@totoro ~ $ /sbin/getpcaps $$
Capabilities for `13976': = cap_net_admin,cap_net_raw+i
  It worked.

So that’s it for this week. Hope to hear from you in the comments :D.

[0] http://www.linuxjournal.com/magazine/making-root-unprivileged

In spite of my exams-period beginning next week, I managed to do something useful for GSoC :).

What did I do this week:
In order to do some better testing I wanted to patch some ebuilds, as I said last week. The question there is, how do you find out which caps the file needs to function? I consulted Friedhoffs page [0] again and downloaded capable_probe. This neat loadable kernel module detects when a program asks for caps and prints it in /var/log/messages. All you need is CONFIG_KPROBES=y in your kernel.
I tried it out and unfortunately it didn’t work. I remembered vaguely, that the caps-format had been changed recently, but I was busy with learning so I didn’t bother to look for the problem (but I came back later :)).
Friedhoff has some example caps-sets on his page and there are also some on [1], so I used these and patched shadow, pam and util-linux.
The patched binaries are:

• passwd - CAP_CHOWN, CAP_DAC_OVERRIDE, CAP_FOWNER
• chsh - CAP_CHOWN, CAP_DAC_READ_SEARCH, CAP_FSETID, CAP_SETUID
• chfn - CAP_CHOWN, CAP_DAC_READ_SEARCH, CAP_FSETID, CAP_SETUID
• chage - CAP_DAC_READ_SEARCH
• expiry - CAP_DAC_OVERRIDE, CAP_SETGID
• gpasswd - CAP_CHOWN, CAP_DAC_OVERRIDE, CAP_SETUID
• newgrp - CAP_DAC_OVERRIDE, CAP_SETGID
• su - CAP_SETGID, CAP_SETUID (this only seems to work, when unix_chkpwd has its cap)
• mount - CAP_DAC_OVERRIDE, CAP_SYS_ADMIN
• umount - CAP_DAC_OVERRIDE, CAP_SYS_ADMIN, (CAP_CHOWN)
• unix_chkpwd - CAP_DAC_OVERRIDE

After testing them, all worked except umount. So to investigate what was missing I turned back to capable_probe. After a bit of digging through kernel docs and sources, the problem was quite clear: The signature of cap_capable had changed (the method which was used to place the Jprobe-hook on). After adapting the signature it worked!

Using capable_probe I detected that umount was asking for CAP_CHOWN, which was not in its set. Because umount failed when trying to move mtab.tmp back to mtab, it seemed reasonable, that CAP_CHOWN could help. And it did :).

All in all I’m very happy today and will write a separate post on capable_probe. I added the patched version to my git-repo [2].

I would be even happier :D if someone would try my patched ebuilds and tell me if all is okay or if there is something missing.

[0] http://www.friedhoff.org/posixfilecaps.html
[1] http://wiki.archlinux.org/index.php/Using_File_Capabilities_Instead_Of_Setuid
[2] http://github.com/constanze/GSoC2010_Gentoo_Capabilities/tree/master/capable_probe/

This week was kind of slow GSoC-wise due to my sister giving birth to my beautiful niece :D. Yes I’m an aunt now.
Moving on to the interesting part – What did I do this week:
I looked into the eclass/tests folder and tried to make a similar “test-suite” for the fcaps eclass.
It contains 4 tests:

1. Unknown capability
2. Known capability
3. Unknown user/group
4. Unknown file-mode

I also compiled a test-kernel without XATTR, to check that possibility and tested if libcap is correctly emerged, if it is not installed, yet. All worked fine.

My second mile-stone, which is due next week looks like this:

June 21th to July 4th
In this period I will work on the USE-Flag support. A lot of testing/integration-testing will be needed here, to make sure the setting of the Capabilities really works. I also want to find out where Capabilities cannot be used (Does it work on all file-systems? Can every setuid be replaced? Which Kernel-Versions can be used?). The user needs to be informed if Capabilities cannot be used on his/her system.

I added a USE-Flag and I did some testing. I also already found out which kernel-settings need to be set and which file-systems support capabilities. The user gets an ewarn if the capabilities could not be set. What I need to find out, but what is not trivial, is if all setuid can be replaced. I think this is something which I have to investigate in, when I begin patching some other ebuild than net-misc/iputils.

In the next week I want to patch some other ebuilds, so I can do better tests with fcaps.

That’s it for today, have a nice week :).

Maybe it’s called summer of code, but summer definitely did not arrive in Munich, yet. We had a lot of rain and temperatures about 13°C all week. But there may be a good side to it, I had no desire to go out – more time to study and code :D.

What did I do this week:
After discussing the further progress of this project with my mentor (flameeyes), we decided, that it would be better to put the fcaps functionality in an eclass, for now. We decided this, because a new eclass does not require an EAPI bump and it would be easier to get fcaps out there ;).
So that’s what I did:

• moving the fcaps stuff to an eclass
• some minor changes considering failure-handling
• adding a USE-flag (filecaps)
• applied an overlay-structure, for easier testing and because it makes more sense to use one

In the next week I will finally do more testing of fcaps.

That’s it for today.
Feel free to write me a comment or mail :).

The next progress report is due and forces me to review what I’ve done.
First I have to admit I have put a lot less time into GSoC than I planned to do. This has to stop. I will do a different schedule for the coming week, let’s see how that turns out.

What did I do this week:
I had a conversation with ferringb about the placement of the fcaps call. As you may know, I was very uncertain about putting it into pkg_postinst and thus messing with the users livesystem. It is evil, but as flameeyes, my mentor put it, not exceedingly evil :D.
So what did we discuss: ferringb made the suggestion to write the info from fcaps to a temp file in src_install, which would then be evaluated in post_pkg_postinst, as a temporary solution until portage supports preserving xattrs/caps. It would still be a hack, but fcaps could be called from src_install.
I looked into this approach and on the way, tried to learn more about how portage handles an emerge. I didn’t got that far, but I hope I understand things a little better now (in any case I know now, that I have a really long road ahead :), thus the definite need to spend much more time on this). I’m very eager to get to know portage better, however I find it a little difficult to get familiar with it. (Any tips?)
On the way of my investigation I also found out that my version of coreutils has a bug [1], so I switched to the unstable one. What I gained is, that now I can copy files and preserve their caps (with –preserve=xattr).
As you can see in my git-repo I didn’t implement ferringbs suggestion. I discussed this approach with flameeyes.

This is what we concluded:

1. This approach would still require logic in postinst to make sure the caps didn’t get lost and the destination file-system supports caps.
2. Handling the caps totally on portage side requires a lot of change and there are a lot of dependencies which might lead to problems.
   To clarify: tools like tar might or might not support preserving the caps, the same goes for coreutils and if I understood correctly python (without add-ons) does not support xattrs altogether. (Please correct me, if you know otherwise)
3. To increase the chance of a new feature to get accepted and used, it is imported to do small steps, which require little change, so it gets out there and doesn’t stay in the development-phase for years.

Personally I think it might be good to keep the hack where you can see it (i.e. in the ebuild) as long as it’s a hack. I get that it would be convenient, if the ebuild could have the fcaps call in the correct place right now, so there is no change required later. But this might also give the false impression, that it’s a clean solution already when it’s still just a hack.

I also added something to fcaps. It now sets owner and file-mode on every file, not just as fallback, but the file-mode with -s of course ;).
This has been added because some binaries like wireshark require other than default owners and/or file-modes.

In the next week I will do more testing of fcaps, which is now in its alpha-version-state (if I forgot to test something, please tell me).
I will also look into how FEATURES are used and implemented as the caps-support might better be triggered by a FEATURE than a use-flag. What I also want to look into is the EAPI-Process, as I’m not sure I got that all correctly. (If you have some links/documentation for me please mail or comment :))

That’s it for today.
Feel free to tell me what you think about fcaps and the process so far and  to make suggestions for a better approach.

[1] http://lists.gnu.org/archive/html/bug-coreutils/2010-03/msg00136.html