coupleprogramming

rulegenerator for firewalls or what I’ve been up to

constanze — Sun, 27 Feb 2011 17:02:15 +0000

Long time no see :). Some of you may have wondered or not, where I disappeared, I’m going to tell you anyway.

I’ve been writing my bachelor thesis and now I’m finally finished :). And I actually developed something useful, or at least, I think it’s useful.

My thesis’ subject is: Design and Prototypic Implementation of a Rule–Generator for Packetfilter–Firewalls with Integrated Detection of Malicious Traffic.

It’s in German, so most of you can’t read it, which you probably wouldn’t want anyway. So I’m going to tell what’s in it.

With your standard personal firewall you have a mechanism, which presents you potential rules, based on connections programs want to establish. This may be unnerving on a windows machine with windows popping up every 5 minutes, but the principle is kind of useful.
What I wanted to do was: Port this mechanism to packetfilter-firewalls. And I did it, it’s not exactly the same, but more or less similar.

I wrote a python program, which can read a traffic dump in pcap format and combines the traffic to rules. The rules can be printed out in pf- and netfilter-syntax. If you wonder: pf is the OpenBSD packetfilter.

Since that was not enough for a bachelor thesis, I also implemented simple portscan and ddos detection algorithms and a mechanism to filter out unwanted rules before printing out the ruleset.

It’s only a prototype and it has problems with traffic-dense dumps (it consumes all your memory and never comes to an end :( ). But all in all it works kind of good, at least in my tests :).

So, you may ask, can I test this rulegenerator-thingy? Yes, you can :D.
I put it on github [1]. You have to have dpkt [2] installed, a “python packet creation / parsing library”, which is used to read the dumps. Don’t forget to read the README :).

And, so you can see how it works right now, I included some examples here. I used the output in pf-syntax, because it’s more compact.

rulegeneration with normal dump

Reading pcap file

Checking for DDoS
Filtering Rules with existing ruleset
Detecting Portscans

Generating TCP-Ruleset
Generating UDP-Ruleset
Generating ICMP-Ruleset

1 rule for protocol tcp from 4 tcp connection(s)
pass none-outer proto tcp from 192.168.1.1 port random to 192.168.1.2 port { 21, 22, 23, 80 }

5 rules for protocol udp from 12 udp connections
pass none-outer proto udp from 192.168.1.2 port 513 to 192.168.1.255 port 513
pass none-outer proto udp from 192.168.1.1 port 53 to 192.168.1.2
pass none-outer proto udp from 192.168.1.2 port 138 to 192.168.1.255 port 138
pass none-outer proto udp from 192.168.1.2 port random to 192.168.1.1 port 53
pass none-outer proto udp from 192.168.1.0/24 port 137 to 192.168.1.255 port 137

No connections for protocol icmp
rulegeneration with dump containing a portscan

Reading pcap file

Checking for DDoS
Detecting Portscans
Detecting infected hosts

Generating TCP-Ruleset
Generating UDP-Ruleset
Generating ICMP-Ruleset

No connections for protocol tcp

1 rule for protocol udp from 1 udp connection(s)
pass none-outer proto udp from 192.168.2.105 port random to 224.0.0.251 port 5353

No connections for protocol icmp

1 rule which results from traffic which looks like a portscan
1000 ports were scanned in range: 1:65389
block none-outer proto tcp from 192.168.2.101 port random to 192.168.2.105
1 IP(s) scanned: 1 IP(s)
192.168.2.101 scanned 192.168.2.105
rulegeneration with dump containing a ddos-attack

Reading pcap file

Checking for DDoS
There might be an tcp (D)DoS Attack. The weighted moving average is 5.41770376331, which is below the given border 400.
Detecting Portscans

Generating TCP-Ruleset
Generating UDP-Ruleset
Generating ICMP-Ruleset

1 rule for protocol tcp from 4000 tcp connection(s)
pass in proto tcp from any port random to 10.0.0.1 port 22

No connections for protocol udp

No connections for protocol icmp

The portscan detection mechanism is a simple “if there are more than x connection/connection-attemps from one host to another host/different ports or other hosts/one port or other hosts/different ports” test.
The ddos detection mechanism is a little bit more complicated. I took it from a paper on ddos-detection [3].

I would be happy if the rulegenerator is of some use to someone out there. If you look at the code, remember it’s just a prototype. If you have suggestions or want to make the rulegenerator better, tell me. Maybe, with a little work, it can be more than just a prototype ;).

[1] https://github.com/constanze/rulegen
[2] http://code.google.com/p/dpkt/
[3] http://www.springerlink.com/content/k6436394vn52275h/

The state of Scala on Gentoo

ntoythi — Sat, 22 Jan 2011 14:45:17 +0000

Scala is an interesting programming language for the Java VM which combines object-oriented with functional programming-style. It’s feature-set is especially attractive for erlang developers (e.g. immutability, functional-style, higher-order-functions) as well as for c++ / boost developers (e.g. closures, first-order-functions, multiple-inheritance via traits). User votes in different java-magazines on future JVM languages clearly see Scala as the language with the highest potential. So it’s worth a look for sure :-)

The current state of Scala on Gentoo Linux is:

There is a dev-lang/scala ebuild for Scala 2.7.7, more than 1 year old (added 30 Nov 2009; current is 2.8.1, ebuild available in Bug 328291)
There is also a dev-java/scala-bin, which is kinda ancient (added 02 Aug 2004, Version 1.2.0.1). I think this ebuild should be removed or updated to 2.8.1.

If we look at the tooling- and library-front it’s even more frustrating. None of these are currently available as an ebuild:

Testing: ScalaTest, ScalaCheck, Specs
Concurrency / Actor framework: Akka
Web-Framework: Lift

I think it’s kinda sad that Scala doesn’t play the role it deserves on Gentoo. The version bump request is open since 14 Jul 2010 and in the meanwhile even the summary is outdated (request was once filed for 2.8.0). I’ll keep posting ebuilds on bugzilla, but I think the community’s interest on Scala is high enough to justify an “official” stable version of a current Scala version in the tree (or does anyone know of such a version in any overlay?)

I know the Gentoo Java Team is understaffed and there are many issues requiring their attention, but I think this would help keeping a lot Java-/Scala developers staying with Gentoo as a development platform. If someone would like to proxy-maintain me I’d be happy to help.

Progress Report #9 for POSIX-Capabilities Project

constanze — Thu, 12 Aug 2010 18:51:27 +0000

So, this is the last progress report of this years GSoC. That doesn’t mean that this project is done, but GSoC has come to an end.

To wrap things up, the version, which will hopefully be added to the tree soon is the one which sets the caps directly in the livefs. Why? Because I still haven’t figured out a good fallback-mechanism, if the livefs doesn’t support caps and the caps can’t be reapplied after the copy.
Don’t worry, I’m working on that, but it won’t be finished by Aug 16th, which is the Pencils Down Date of GSoC. So flameeyes and me decided, that it would be good, to see this version as a final result.

Thanks you all for trying my stuff and commenting. This was the best summer of my life so far, I learned a lot and had a great time. Thanks to my mentor Diego E. “Flameeyes” Pettenò for giving me feedback and encouraging me, when I wasn’t so sure I’m on the right path.

As for my involvement with Gentoo, there is already a recruitment bug for me in bugzilla ;).

Progress Report #8 for POSIX-Capabilities Project

constanze — Sat, 31 Jul 2010 12:05:15 +0000

Again, it has been two weeks since my last report, shame on me. But my exams are all written by now (and passed :D) and I have some exciting new stuff to tell you :).

What did I do this week (and the last):
The next step in this project was to find out why and where the caps get lost, when they are set in src_install (that means on the binary in the sandbox and not on the binary which is already in the livefs).
After a long and sometimes frustrating journey through the portage sources and with a lot help from pdb I found out why.
Normally (e.g. when the source and the destination are on the same filesystem) os.rename is used to move the binary from sandbox to livefs. This happens in portage/pym/portage/util/movefile.py and is not were the caps are lost.
They are lost earlier, when the binaries are stripped. So I added a preserving-mechanism to portage/bin/ebuild-helpers/prepstrip, where the stripping happens. And it worked :). I could set the caps in src_install and they did not get lost.
But there was another place where the caps could get lost. If os.rename in movefile.py fails or src and dest are not on the same filesystem, shutil.copy is used before os.rename and the binarys mode/ownership are restored after. Of course the extended attributes, like caps, have to be restored, too. Using pyxattr I extracted the caps before the copy and applied them back after the binary is copied and its mode/ownership are restored. The downside is, that portage now needs pyxattr.
With that now working, I changed all ebuilds I have patched so far, so the caps are set in src_install.
I added the changed ebuilds and the changed portage-files to a new branch in my repository. I used the current stable version (2.1.8.3) of portage, just so you know :).

That’s it for today, tell me what you think in the comments :).

Progress Report #7 for POSIX-Capabilities Project

constanze — Fri, 16 Jul 2010 20:00:12 +0000

This report is a little late, but I’m in my exams period and had to do a lot of studying. I had three exams already and have two to go next week, so I will be back full time, after that. This report is about this and the last week, enjoy :D.

What did I do this week (and the last):
Since some “easy” ebuilds have been patched, flameeyes suggested, I could take a look at tcpdump. While tcpdump has no suid-bit set, it can only be run as root, which is inconvenient. It would be nice if “special” users could run it to sniff (without using sudo) and normal users can use it to view dumps. Then it could also be moved to /usr/bin/. So how could you realize that with caps?
There is a pam module called pam_cap, which allows you to put caps in the users inherited set.
Then there are two options:

Is your application capability-aware?
You can put the caps in the permitted set and/or inherited set.
Is your application not capability-aware?
You will have to put the caps in the inherited and effective set.

Only capability-aware application can populate their effective set themselves, the kernel has to do it for the others. And the kernel knows, if the effective set is not empty, the application is not capability-aware [0].

So, how do you use pam_cap?

You need to have libcap installed. (Obviously :))
You need to include pam_cap.so in your /etc/pam.d/system-auth.
Something like this:
auth required pam_env.so auth required pam_cap.so ...
Add caps to users in /etc/security/capability.conf.
Like this:
#netadmin inherits cap_net_admin,cap_net_raw cap_net_admin,cap_net_raw netadmin #every other user inherits nothing none *
The last line is very important, otherwise all other users inherit all caps, which would be not what we wanted.
Logout/Login
It should work now, but maybe you need to reboot.

Now, that you can grant users certain caps, you can put the caps on your binaries in ei or p/pi instead of ep and only the users, who have the right caps in their set can execute these binaries correctly. To try it out you can emerge tcpdump from my overlay, which has cap_net_raw in its effective and inherited set. Only a user who has at least cap_net_raw in his/her set can sniff with tcpdump.

You want to see this as an example, before you try it out on your system? Here you go:
This is my /etc/pam.d/system-auth:

auth required pam_env.so
auth required pam_cap.so
auth required pam_unix.so try_first_pass likeauth nullok

account required pam_unix.so

password required pam_cracklib.so difok=2 minlen=8 dcredit=2 ocredit=2 retry=3
password required pam_unix.so try_first_pass use_authtok nullok sha512 shadow
session required pam_limits.so
session required pam_env.so
session required pam_unix.so
session optional pam_permit.so

This is my /etc/security/capabiliy.conf

#
# /etc/security/capability.conf
#
# this is a sample capability file (to be used in conjunction with
# the pam_cap.so module)
#
# In order to use this module, it must have been linked with libcap
# and thus you’ll know about Linux’s capability support.
# [If you don't know about libcap, the sources for it are here:
#
# http://linux.kernel.org/pub/linux/libs/security/linux-privs/
#
# .]
#
# Here are some sample lines (remove the preceding ‘#’ if you want to
# use them

## user ‘morgan’ gets the CAP_SETFCAP inheritable capability
##cap_setfcap morgan

cap_net_admin,cap_net_raw netadmin

## ‘everyone else’ gets no inheritable capabilities
none *

## if there is no ‘*’ entry, all users not explicitly mentioned will
## get all available capabilities. This is a permissive default, and
## probably not what you want…

tcpdump is now in /usr/bin/, let’s look at its caps.superkfr@totoro ~ $ sudo getcap /usr/bin/tcpdump /usr/bin/tcpdump = cap_net_raw+ei
It has cap_net_raw in the effective and inherited set, good so far.

Let’s try to execute tcpdump with the user netadmin, who has the required caps.
superkfr@totoro ~ $ su - netadmin Password: netadmin@totoro ~ $ tcpdump -i wlan0 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on wlan0, link-type EN10MB (Ethernet), capture size 65535 bytes ... netadmin@totoro ~ $ exit logout
Works just fine.

Let’s try with user test, who has no caps inherited.
superkfr@totoro ~ $ su - test Password: test@totoro ~ $ tcpdump -i wlan0 tcpdump: wlan0: You don't have permission to capture on that device (socket: Operation not permitted) test@totoro ~ $ tcpdump -r /tmp/dump reading from file /tmp/dump, link-type EN10MB (Ethernet) ... test@totoro ~ $ exit logout
Like we expected, user test does not have the right permissions to sniff, but can view dumps.

If you have trouble getting pam_cap to work (believe me, I’ve been there), try this:
Look at the output of /sbin/getpcaps $$ in the shell of the user, whom you want to have the caps.

If the output looks something like this:
test@totoro ~ $ /sbin/getpcaps $$ Capabilities for `8219': =
The users cap set is empty and something didn’t work. Try rebooting, if you haven’t done this already.
If the output looks something like this:
netadmin@totoro ~ $ /sbin/getpcaps $$ Capabilities for `13976': = cap_net_admin,cap_net_raw+i
It worked.

So that’s it for this week. Hope to hear from you in the comments :D.

[0] http://www.linuxjournal.com/magazine/making-root-unprivileged

Detecting needed capabilities with capable_probe

constanze — Mon, 05 Jul 2010 11:24:29 +0000

If you want to find out which capabilities a binary needs, you have several options:

You use strace and determine them from the output
You use capable_probe

The first method is quite simple, but you have to deduct the caps from the error messages, which can be a bit difficult.
The second method is more convenient and simple, so I’m going to describe this method further.
capable_probe is a loadable kernel module developed by Serge Hallyn [0]. It uses the kprobe kernel mechanism [1] to insert a Jprobe. And what does it do exactly?

When this kernel module is inserted, any calls to cap_capable() are replaced by a call to the cr_capable() function. This function prints the name of the program that requires capabilities and the capability being checked. It then continues executing the actual cap_capable() call through the call to jprobe_return(). [0]

The signature of cr_capable has to be the same as the signature of cap_capable. The signature changed from
int cap_capable (struct task_struct *tsk, int cap)
to
int cr_capable (struct task_struct *tsk, const struct cred *cred, int cap, int audit).
So to get capable_probe to work you need to change the signature or use the one from my repository [2].

There is a good README by Chris Friedhoff [3], which describes how to install and use it.

Important: Don’t forget to unload capable_probe after you determined the caps, as it will spam your /var/log/messages extremely.

To see capable_probe in action here is how to determine the caps for passwd:

Remove suid
chown -s /bin/passwd
tail -f /var/log/messages | grep passwd
modprobe capable_probe
Try to change your password
This is what you get from passwd:
superkfr@totoro ~ $ passwd Changing password for superkfr. (current) UNIX password: Password: Retype new password: passwd: Authentication token lock busy passwd: password unchanged Not very informative.
modprobe -r capable_probe
Let’s see what we got from capable_probe:
Jul 5 15:20:54 totoro kernel: cr_capable: asking for capability 1 for passwd Jul 5 15:20:54 totoro kernel: cr_capable: asking for capability 2 for passwd Jul 5 15:20:54 totoro kernel: cr_capable: asking for capability 1 for passwd Jul 5 15:20:54 totoro kernel: cr_capable: asking for capability 2 for passwd Jul 5 15:20:54 totoro kernel: cr_capable: asking for capability 24 for passwd
What does this all mean?
The numbers stand for different caps, you can look them up in /usr/include/linux/capability.h.

/* Override all DAC access, including ACL execute access if [_POSIX_ACL] is defined. Excluding DAC access covered by CAP_LINUX_IMMUTABLE. */

#define CAP_DAC_OVERRIDE 1

/* Overrides all DAC restrictions regarding read and search on files and directories, including ACL restrictions if [_POSIX_ACL] is defined. Excluding DAC access covered by CAP_LINUX_IMMUTABLE. */

#define CAP_DAC_READ_SEARCH 2 ...

/* Override resource limits. Set resource limits. */ /* Override quota limits. */ /* Override reserved space on ext2 filesystem */ /* Modify data journaling mode on ext3 filesystem (uses journaling resources) */ /* NOTE: ext2 honors fsuid when checking for resource overrides, so you can override using fsuid too */ /* Override size restrictions on IPC message queues */ /* Allow more than 64hz interrupts from the real-time clock */ /* Override max number of consoles on console allocation */ /* Override max number of keymaps */

#define CAP_SYS_RESOURCE 24
Now, which caps seem plausible? passwd needs to read and write /etc/shadow so it needs CAP_DAC_OVERRIDE, CAP_DAC_READ_SEARCH is a subset, is does not have to be set additionally. There is no indication, that passwd could need CAP_SYS_RESOURCE. So we try it out with CAP_DAC_OVERRIDE.
sudo setcap cap_dac_override=ep /bin/passwd sudo modprobe capable_probe passwd
And it fails again.
Jul 5 16:02:50 totoro kernel: cr_capable: asking for capability 1 for passwd Jul 5 16:02:50 totoro kernel: cr_capable: asking for capability 2 for passwd Jul 5 16:02:50 totoro kernel: cr_capable: asking for capability 1 for passwd Jul 5 16:02:50 totoro kernel: cr_capable: asking for capability 24 for passwd Jul 5 16:03:01 totoro kernel: cr_capable: asking for capability 1 for passwd Jul 5 16:03:01 totoro kernel: cr_capable: asking for capability 1 for passwd Jul 5 16:03:01 totoro kernel: cr_capable: asking for capability 1 for passwd Jul 5 16:03:01 totoro kernel: cr_capable: asking for capability 0 for passwd Jul 5 16:03:01 totoro kernel: cr_capable: asking for capability 1 for passwdIt also asks for CAP_CHOWN(0) now, that seems also plausible, so grant it and repeat.
It fails again, what is still missing?
Jul 5 16:03:56 totoro kernel: cr_capable: asking for capability 1 for passwd Jul 5 16:03:56 totoro kernel: cr_capable: asking for capability 2 for passwd Jul 5 16:03:56 totoro kernel: cr_capable: asking for capability 1 for passwd Jul 5 16:03:56 totoro kernel: cr_capable: asking for capability 24 for passwd Jul 5 16:04:06 totoro kernel: cr_capable: asking for capability 1 for passwd Jul 5 16:04:06 totoro kernel: cr_capable: asking for capability 1 for passwd Jul 5 16:04:06 totoro kernel: cr_capable: asking for capability 1 for passwd Jul 5 16:04:06 totoro kernel: cr_capable: asking for capability 0 for passwd Jul 5 16:04:06 totoro kernel: cr_capable: asking for capability 0 for passwd Jul 5 16:04:06 totoro kernel: cr_capable: asking for capability 3 for passwd Jul 5 16:04:06 totoro kernel: cr_capable: asking for capability 1 for passwdIt also wants CAP_FOWNER(3)
Add that to the set and it works.
Don’t forget:
modprobe -r capable_probe

This example is a little more complicated than the standard ping-example, but I wanted to do a different one. Hope that helps :).

[0] http://www.ibm.com/developerworks/library/l-posixcap.html [1] You need CONFIG_KPROBES=y in your kernel [2] http://github.com/constanze/GSoC2010_Gentoo_Capabilities/tree/master/capable_probe/ [3] http://www.friedhoff.org/posixfilecaps.html

Progress Report #6 for POSIX-Capabilities Project

constanze — Sun, 04 Jul 2010 18:27:22 +0000

In spite of my exams-period beginning next week, I managed to do something useful for GSoC :).

What did I do this week:
In order to do some better testing I wanted to patch some ebuilds, as I said last week. The question there is, how do you find out which caps the file needs to function? I consulted Friedhoffs page [0] again and downloaded capable_probe. This neat loadable kernel module detects when a program asks for caps and prints it in /var/log/messages. All you need is CONFIG_KPROBES=y in your kernel.
I tried it out and unfortunately it didn’t work. I remembered vaguely, that the caps-format had been changed recently, but I was busy with learning so I didn’t bother to look for the problem (but I came back later :)).
Friedhoff has some example caps-sets on his page and there are also some on [1], so I used these and patched shadow, pam and util-linux.
The patched binaries are:

passwd - CAP_CHOWN, CAP_DAC_OVERRIDE, CAP_FOWNER
chsh - CAP_CHOWN, CAP_DAC_READ_SEARCH, CAP_FSETID, CAP_SETUID
chfn - CAP_CHOWN, CAP_DAC_READ_SEARCH, CAP_FSETID, CAP_SETUID
chage - CAP_DAC_READ_SEARCH
expiry - CAP_DAC_OVERRIDE, CAP_SETGID
gpasswd - CAP_CHOWN, CAP_DAC_OVERRIDE, CAP_SETUID
newgrp - CAP_DAC_OVERRIDE, CAP_SETGID
su - CAP_SETGID, CAP_SETUID (this only seems to work, when unix_chkpwd has its cap)
mount - CAP_DAC_OVERRIDE, CAP_SYS_ADMIN
umount - CAP_DAC_OVERRIDE, CAP_SYS_ADMIN, (CAP_CHOWN)
unix_chkpwd - CAP_DAC_OVERRIDE

After testing them, all worked except umount. So to investigate what was missing I turned back to capable_probe. After a bit of digging through kernel docs and sources, the problem was quite clear: The signature of cap_capable had changed (the method which was used to place the Jprobe-hook on). After adapting the signature it worked!

Using capable_probe I detected that umount was asking for CAP_CHOWN, which was not in its set. Because umount failed when trying to move mtab.tmp back to mtab, it seemed reasonable, that CAP_CHOWN could help. And it did :).

All in all I’m very happy today and will write a separate post on capable_probe. I added the patched version to my git-repo [2].

I would be even happier :D if someone would try my patched ebuilds and tell me if all is okay or if there is something missing.

[0] http://www.friedhoff.org/posixfilecaps.html
[1] http://wiki.archlinux.org/index.php/Using_File_Capabilities_Instead_Of_Setuid
[2] http://github.com/constanze/GSoC2010_Gentoo_Capabilities/tree/master/capable_probe/

Progress Report #5 for POSIX-Capabilities Project

constanze — Sun, 27 Jun 2010 17:17:32 +0000

This week was kind of slow GSoC-wise due to my sister giving birth to my beautiful niece :D. Yes I’m an aunt now.
Moving on to the interesting part – What did I do this week:
I looked into the eclass/tests folder and tried to make a similar “test-suite” for the fcaps eclass.
It contains 4 tests:

Unknown capability
Known capability
Unknown user/group
Unknown file-mode

I also compiled a test-kernel without XATTR, to check that possibility and tested if libcap is correctly emerged, if it is not installed, yet. All worked fine.

My second mile-stone, which is due next week looks like this:

June 21th to July 4th
In this period I will work on the USE-Flag support. A lot of testing/integration-testing will be needed here, to make sure the setting of the Capabilities really works. I also want to find out where Capabilities cannot be used (Does it work on all file-systems? Can every setuid be replaced? Which Kernel-Versions can be used?). The user needs to be informed if Capabilities cannot be used on his/her system.

I added a USE-Flag and I did some testing. I also already found out which kernel-settings need to be set and which file-systems support capabilities. The user gets an ewarn if the capabilities could not be set. What I need to find out, but what is not trivial, is if all setuid can be replaced. I think this is something which I have to investigate in, when I begin patching some other ebuild than net-misc/iputils.

In the next week I want to patch some other ebuilds, so I can do better tests with fcaps.

That’s it for today, have a nice week :).

Progress Report #4 for POSIX-Capabilities Project

constanze — Sun, 20 Jun 2010 18:26:58 +0000

Maybe it’s called summer of code, but summer definitely did not arrive in Munich, yet. We had a lot of rain and temperatures about 13°C all week. But there may be a good side to it, I had no desire to go out – more time to study and code :D.

What did I do this week:
After discussing the further progress of this project with my mentor (flameeyes), we decided, that it would be better to put the fcaps functionality in an eclass, for now. We decided this, because a new eclass does not require an EAPI bump and it would be easier to get fcaps out there ;).
So that’s what I did:

moving the fcaps stuff to an eclass
some minor changes considering failure-handling
adding a USE-flag (filecaps)
applied an overlay-structure, for easier testing and because it makes more sense to use one

In the next week I will finally do more testing of fcaps.

That’s it for today.
Feel free to write me a comment or mail :).

Progress Report #3 for POSIX-Capabilities Project

constanze — Sun, 13 Jun 2010 19:53:58 +0000

The next progress report is due and forces me to review what I’ve done.
First I have to admit I have put a lot less time into GSoC than I planned to do. This has to stop. I will do a different schedule for the coming week, let’s see how that turns out.

What did I do this week:
I had a conversation with ferringb about the placement of the fcaps call. As you may know, I was very uncertain about putting it into pkg_postinst and thus messing with the users livesystem. It is evil, but as flameeyes, my mentor put it, not exceedingly evil :D.
So what did we discuss: ferringb made the suggestion to write the info from fcaps to a temp file in src_install, which would then be evaluated in post_pkg_postinst, as a temporary solution until portage supports preserving xattrs/caps. It would still be a hack, but fcaps could be called from src_install.
I looked into this approach and on the way, tried to learn more about how portage handles an emerge. I didn’t got that far, but I hope I understand things a little better now (in any case I know now, that I have a really long road ahead :), thus the definite need to spend much more time on this). I’m very eager to get to know portage better, however I find it a little difficult to get familiar with it. (Any tips?)
On the way of my investigation I also found out that my version of coreutils has a bug [1], so I switched to the unstable one. What I gained is, that now I can copy files and preserve their caps (with –preserve=xattr).
As you can see in my git-repo I didn’t implement ferringbs suggestion. I discussed this approach with flameeyes.

This is what we concluded:

This approach would still require logic in postinst to make sure the caps didn’t get lost and the destination file-system supports caps.
Handling the caps totally on portage side requires a lot of change and there are a lot of dependencies which might lead to problems.
To clarify: tools like tar might or might not support preserving the caps, the same goes for coreutils and if I understood correctly python (without add-ons) does not support xattrs altogether. (Please correct me, if you know otherwise)
To increase the chance of a new feature to get accepted and used, it is imported to do small steps, which require little change, so it gets out there and doesn’t stay in the development-phase for years.

Personally I think it might be good to keep the hack where you can see it (i.e. in the ebuild) as long as it’s a hack. I get that it would be convenient, if the ebuild could have the fcaps call in the correct place right now, so there is no change required later. But this might also give the false impression, that it’s a clean solution already when it’s still just a hack.

I also added something to fcaps. It now sets owner and file-mode on every file, not just as fallback, but the file-mode with -s of course ;).
This has been added because some binaries like wireshark require other than default owners and/or file-modes.

In the next week I will do more testing of fcaps, which is now in its alpha-version-state (if I forgot to test something, please tell me).
I will also look into how FEATURES are used and implemented as the caps-support might better be triggered by a FEATURE than a use-flag. What I also want to look into is the EAPI-Process, as I’m not sure I got that all correctly. (If you have some links/documentation for me please mail or comment :))

That’s it for today.
Feel free to tell me what you think about fcaps and the process so far and to make suggestions for a better approach.

[1] http://lists.gnu.org/archive/html/bug-coreutils/2010-03/msg00136.html