I`m not the biggest fan of openSUSE but this weeks post by Colin Charles makes me happy ..

openSUSE users can now do a mariadb install from their default repositories.

With all the fuzz about Snoracle and MySQL's future last year to me it became clear that we would end up having different MySQL based distributions, probably with different names, and that it would be up to the Linux distributions to provide the users with what they preferred, working with those Linux distributions
therefore would be very important for the MySQL distributions.

Sadly my Fedora box doesn't allow me to do a yum install mariadb yet ... but I`m sure that's only a matter of time ..

Technorati Tags: distributions fedora mariadb mysql opensuse snoracle Share with Shareomatic! --> Trackback URL for this post: http://www.krisbuytaert.be/blog/trackback/1014

After having setup squid and dansguardian (using clamd) on Centos 5, I wasn't able to use it :(

I had always the following error, even if the dansguardian user was the same as clamd (clamav) :

2010.7.9 12:22:41 - 10.0.200.6 http://www.eicar.org/anti_virus_test_file.htm *INFECTED* *DENIED* /tmp/tfIlR1j6: lstat() failed: Permission denied. ERROR GET 15590 0 Content scanning 1 403 text/html

I just realize after having searched too long that SELinux (I know life is too short for it) was the culprit.
It was my mistake as I completely forgot that this machine had selinux enabled :-S

So in /var/log/audit/audit.log I had :

type=AVC msg=audit(1278673113.470:3489): avc: denied { getattr } for pid=32164 comm="clamd" path="/tmp/tfCSCirx" dev=dm-3 ino=17 scontext=user_u:system_r:clamd_t:s0 tcontext=user_u:object_r:initrc_tmp_t:s0 tclass=file type=SYSCALL msg=audit(1278673113.470:3489): arch=c000003e syscall=6 success=no exit=-13 a0=8cce370 a1=421f2dc0 a2=421f2dc0 a3=8 items=0 ppid=1 pid=32164 auid=1004 uid=102 gid=114 euid=102 suid=102 fsuid=102 egid=114 sgid=114 fsgid=114 tty=(none) ses=437 comm="clamd" exe="/usr/sbin/clamd" subj=user_u:system_r:clamd_t:s0 key=(null)

Note to myself: Never forget to check in audit.log !

To create the selinux policies, I used the following commands, which are quiet easy:

audit2allow -a -m dansguardian > dansguardian.te checkmodule -M -m dansguardian.te checkmodule -M -m dansguardian.te -o dansguardian.mod semodule_package -o dansguardian.pp -m dansguardian.mod semodule -i dansguardian.pp

Et voilà ! Dansguardian is running and I didn't disable selinux :-)

I've been fighting several weeks (and making a huge number of typo's due to that) with the touchpad of my macbook pro on Fedora/Gnome.

I've tested several solutions :

- disable it in Gpointing Device Settings --> fail (it always comes back after a short moment)
- use synclient TouchpadOff=1 --> fail
- creating udev rules : --> fail

ACTION=="add", SUBSYSTEM=="input", ENV{ID_CLASS}="mouse", RUN+="/usr/bin/synclient TouchpadOff=1" ACTION=="remove", SUBSYSTEM=="input", ENV{ID_CLASS}="mouse", RUN+="/usr/bin/synclient TouchpadOff=0

So the best solution I've found (one that works) is : rmmod bcm5974

Now I'll try to add it into the udev rules too.

What happens when you mention Open Office and Firewall in once sentence, in public ?

People start actually building it (French Article)

Then add to that list that there's also people out there that think that running MySQL over NFS is providing them High Availability, or that using DNS Round Robin will provide them a scalable setup,

So yes .. apparently there is indeed a parallel universe out there.

And no .. I don't want to see Webmin in any Appliance .. that is a joke..., or rather a rant ..

Technorati Tags: firewall open source openoffice presentation weridows zarafa zarafacamp zarafasummercamp Share with Shareomatic! --> Trackback URL for this post: http://www.krisbuytaert.be/blog/trackback/1013

Couple of Fridays ago we had one of our @Inuits days again. Rather than having some people give talks and presentations about what they have been doing for the past couple of months this time we set out to research, test, and build stuff.

We split up in 3 different groups, one focusing on CI and testing freshly build stuff with cucumber, a second one setup and tested Galera

We setup a 3 node Galera cluster , not really as smooth as we'd like to ..

Our first bump was that the installation of the package on CentOS is hell, it needs manual interaction such as replacing packages. Deploying this from a repository is probably not going to be a straight forward option.

Galera only takes care of replicating data, just as with MySQL MM replication there still is a need for an external tool to define where to access the database, and implement monitoring in such a way that you are connecting to an up to date database.

Karl started wondering about Galera's locking, turns out the locks aren't cluster wide, locks within the same node work fine.. so if galera is solely used for HA with 1 active node and X failover nodes, it will work (so all transactions happening on 1 node).

We also ran into some issues when trying to start a node which couldn't contact the wsrep_cluster_address point (which is a node it will sync from at startup if specified in the wsrep.cnf file) , it just didn't want to start. This means that when the referenced node (configured in wsrep_cluster_address)is down, you will need to comment it out before you are able to start the mysql server.

The fact that Galera replicates everythying brought us to the discussion if we really wanted that , or if we wanted more finegrained control over which databases or even tables we want to replicate and which ones we didn't want to replicate. A minority of people wanted to replicate everything, the majority of our group wanted finere grained control over what is being replicated to another node.

I`m sure Lefred will shortly be writing about the progress his group made on Banquise

The day ended as it should .. with BBQ and plenty of drinks

Technorati Tags: banquise ci galera inuits inuits day mysql Share with Shareomatic! --> Trackback URL for this post: http://www.krisbuytaert.be/blog/trackback/1012

This morning my HTC Hero told me it had an upgrade available.
It wasn't really the moment to do the upgrade.

So when tonight I wanted to perform the upgrade I couldn't really find out how to initiate it again .

Apparently the trick is to put the date of your phone one month forward and you get the update request again.

So my phone has been updated ... so let's hope this indeed was the preparation for a real upgrade ..

Oh and don't forget to put it back to the original date :)

Technorati Tags: android hero ota upgrade Share with Shareomatic! --> Trackback URL for this post: http://www.krisbuytaert.be/blog/trackback/1011

As you might have read on my Twitter, I got married two weeks ago. My wife – it still feels strange to say – is the most wonderful girl on this planet, and far beyond. Our marriage really was a day which i will remember for the rest of my life. Everything was perfect. [...]

It's a typical situation, the developers develop on their own boxen, they only start to integrate their code on on the production platform 3 hours before the deadline. And then the problems start, the typical "But it works on my system" , "its your problem now" is something nobody really likes to hear .

So how do you tackle this problem ? As Christian already mentions Talking is the first step of the solution,

But one of the most satisfying approaches to solve this problem is to provide your development teams with a standard platform that you support, and a platform they can play with , if you can't provide them with a fully defined platform, give them a set of guide lines on what they can expect. Things like library versions, database types , memory availability and storage availability are key components of such guidelines.

My platform of choice for this kind of projects today is to for an Enterprise Level distro, a distro that stays stable for a longer period, not one that is bleeding edge and changes every other week. So a CentOS or a Debian based distro is probably going to be the platform of choice. But a stable standard platform also means that all the latest nice features a developer wants to have from the bleeding edge libraries he is using aren't going to be available .

Sometimes your devs really need those features, sometimes its just a nice to have. On the other hand you as an ops guy don't want to be packaging and configurating every single tool they dream off. As usual in a Devops environment the key can be found in communication ... Talking with the devs will teach you what features they really need and how they might solve things in a different, more standardized way

We've learned that by giving them a default platform and keeping an open conversation helps, some developers take longer to understand the process others jump in right away .. but in the long term you really need to talk to your devs as soon as possible when they think of implementing a new project that has to run on your platorms.

Lets you sleep at night ..

Technorati Tags: centos devops platform standardize Share with Shareomatic! --> Trackback URL for this post: http://www.krisbuytaert.be/blog/trackback/1010

Yesterday @beaker posted his ideas on the #devops movement ...

Apparently we haven't been stressing enough on the fact that it isn't just about Devs and Ops,
So let me repeat it's not just about Devs and Ops, it's about breaking silo's , about being good at our jobs, about getting conversation started, about talking to different stakeholders in the processes . We are absolutely trying to include all groups, not exclude some.

@beaker also seems to have seen many presentations where developers are shown to have evolved in practice and methodology, but operators (of all kinds) are described as being stuck in the dark ages. , is that a different point of view on another continent \, on this side of the Atlantic, it's mostly the Ops people that are already using agile methods spreading the word and it isn't about Devs talking about Deopvs yet. It's actually mostly the ops spreading the word because they feel most of the pain .

Hoff also wonders about routers switches firewall and all the other boxen where we aren't running puppet or chef on , the boxes that are left out of our fully automated environments .
Indeed, Puppetcamp Europe once again woke up the discussion on how to tackle these boxen, the lack of use of existing standards was covered .. and some mentioned that CIM and family are pretty much death or irrelevant for real life usage , both the Puppet and Chef communities are working on manifest, modules and recipes to solve the issues.

But the good thing is that we now have the security people involved too, maybe they'll figure out how to survive longer than 6 months in a CSO position if they talk to the others and come out of their Ivory towers :)

Technorati Tags: cfengine chef cim devops dtmf puppet SecOPS Share with Shareomatic! --> Trackback URL for this post: http://www.krisbuytaert.be/blog/trackback/1008

Last week was pretty heavy on conferences for me. On wednesday I had to give my Building Virtual Appliances talk at the at the Sizing Server event on Advanced Virtualization and Hybrid Cloud Computing , but the most important part of the week was the first edition of Puppetcamp Europe.

When the first ideas about PuppetCamp Europe started I asked Luke when and where it'd be held. He replied that I should know as I was supposed to organise it... I thanked for the honour , he went on to ask Patrick , he accepted ... I hope I helped him out enough :) I even handed out a personal invitation to some of the most famous configuration mgmt people on this planet and Inuits sponsored the event too

Luke started with the opening talk, talking about the future and past of puppet , about version numbers, 2.6 does sound familiar and stable doesn't it, about forge.puppetlabs.com
During @puppetmasterd 's talk @kartar played Bugmaster which was great and almost realtime

The real fun started with the Open Spaces ... after everybody presented themselves, a mix of usual suspects, first timers and oldskoolers from irc #puppet that finally got faces, different sessions were proposed, ranging from Puppet 101, Alternative Puppet Architectures, Puppet HA, MultiMaster Puppet to Dating for PuppetMasters

Over the 2 days spread the open space different ideas came up on e.g how to scale puppet. Different people are letting their puppetclients run from cron in batches, but probably the weirdest idea I heard was to run Puppet in Jruby in order to speed it up.

Lots of talk on certificates and how to solve the pains with them .. e.g like in a HA setup .. you need to create an authority chain .. there was also talk about having a
--trust-my-network feature that would disable certificates, Luke was open to accepting such a patch, or a patch that would make the whole certificate setup more pluggable
That would for sure be a feature a lot of people would want to use ..

The thurday evening conference dinner was "Stoofvlees met Frieten" for most of us .. but for me it was a London Devops Curry in Gent, with @unixdaemon @ripienaar and some others ;)

But with lots of interesting chatter, free beer and free icecream there's for sure going to be another similar event in Europe next year ..

Technorati Tags: configuration mgmt curry devops ghent jruby puppet puppetcamp ruby zebrastraat Share with Shareomatic! --> Trackback URL for this post: http://www.krisbuytaert.be/blog/trackback/1007

For all the security experts : the NLUUG has published it's Call For Abstracts for it's Fall conference.. as you might have guessed the topic is Security, we welcome all abstracts tackling security in a broad sense.

Possible topics include:

* cloud security
* online privacy
* rfid hacking
* secure programming
* programma-analysis-tools
* web services security
* web browser security
* embedded hardware hacking
* incident response and forensics
* malware and rootkits
* responsible disclosure
* legal response
* fighting spam
* patch policies
* identity management
* central point of administration
* DNSsec
* VPN based WANs
* etc.

The NLUUG fall conference is scheduled on 11 November 2010 in De Reehorst in Ede, the Netherlands.

Hint.. maybe a talk on secdevops would be welcomed too :)

Disclaimer : I`m on the program committee

Technorati Tags: cfp conference devops nluug secdevops SecOPS security Share with Shareomatic! --> Trackback URL for this post: http://www.krisbuytaert.be/blog/trackback/1006

It’s been a while …

A little late to report on some things, but still, here we go.

Loadays
Was only able to make it one day, Saturday, but it was good!
Really enjoyed it. Rather ’small’ conf ( guess bigger then expected though with even some international speakers!
Devops! Talks on system deployment, config management ( we had ‘em all, cfengine, chef and puppet), monitoring …

NLUUG Spring conference
Kris, Kenny and I drove to Holland to attend this anual conference. Main line through this conf was systems administration. Although we spotted some interesting topics, we found them to be not enough in depth.
Eg: the asterisk talk was more like a story about the fact that the speaker and his company implemented asterisk at a client, but didn’t really cover asterisk itself, the part we were interested in …

Puppetcamp
And the last one in row.
Two days on just 1 topic … isn’t that too much?
Well apparently: no it isn’t! Maybe a bit short?
Only attended the first day.
Day schema: talks before noon and open spaces during the after noon.
I really liked Luke’s talk (http://www.slideshare.net/lkanies/portable-infrastructure-with-puppet), the first one of the day.
First part was about upcoming versions, new features etc.
Second part about why puppet exists.
Although Luke mentioned having slept only for 2 hours, he managed to give an interesting and entertaining talk!
The afternoon was filled with openspace slots across the different rooms.
I’ve never ‘openspaced’ before, so I admit: the idea of people walking in and out of rooms, prolly making their point and then just leave … I wasn’t sure this would go somewhere.
But in the end it all turns out very well
Went to the puppet 101 openspace. This ended in a public-driven hands-on overview. Great!

I used preupgrade to updgrade Fedora from 12 to 13.

After the process, I had to resync the partition in refit to be able to boot Linux.

I rebuilded the needed packages for nvidia and the broadcom wireless card.

I needed also to do some modifications to be able to use the integrated iSight webcam:

1. download the apple firmware :

wget http://www.i-nz.net/files/projects/linux-kernel/isight/against-revision-...

then extract it (using the Fedora 12 package isight-firmware-tools) :

[root@delvaux ~]# su -c "ift-extract --apple-driver AppleUSBVideoSupport"

after this operation it should be working for most of the macbook pro, but not for this model, another change is needed.

first find the idProduct number:

[fred@delvaux Desktop]$ lsusb -v | grep iSight -B 3 | grep idProduct idProduct 0x8507

and mofify the file /etc/udev/rules.d/isight.rules with the returned value :

[root@delvaux rules.d]# cat isight.rules ACTION=="add", SYSFS{idVendor}=="05ac", SYSFS{idProduct}=="8507", RUN+="/usr/lib64/udev/ift-load --firmware /lib/firmware/isight.fw" AttachmentSize kmod-wl-2.6.33.4-95.fc13.x86_64-5.10.91.9.3-3.fc13.11.x86_64.rpm540.61 KB kmod-wl-5.10.91.9.3-3.fc13.11.x86_64.rpm6.91 KB kmod-nvidia-2.6.33.4-95.fc13.x86_64-195.36.24-1.fc13.5.x86_64.rpm3.39 MB kmod-nvidia-195.36.24-1.fc13.5.x86_64.rpm29.6 KB

Johan from Sizing Servers asked me if I could talk about my experiences on building (virtual) appliances at their Advanced Virtualization and Hybrid Cloud seminar . Off course I said yes ..

Slides are below ... Enjoy ..

Building appliances View more presentations from Kris Buytaert. Technorati Tags: automation devops kvm virtualbox virtualization xen Share with Shareomatic! --> Trackback URL for this post: http://www.krisbuytaert.be/blog/trackback/1005

Following up on Wim's example

Technorati Tags: buytaert drupal mollom opensource Share with Shareomatic! --> Trackback URL for this post: http://www.krisbuytaert.be/blog/trackback/1004

I'm currently deploying redmine for a customer, and today we ran into a strange issue.

People were able to login, but for certain operations some of them get an "Invalid form authenticity token" error. Moreover redmine was setting more that one cookie with different values and paths in firefox. After some time I figure out that RAILS_RELATIVE_URL_ROOT was set in the apache configuration but was empty. It looks like firefox and IE behave differently if the path of the cookie is empty, firefox considers that the path is the current directory and IE thinks it's '/' Now everything seems working.

I will try to blog a little more about what I'm doing at work

Last week the NLUUG Spring Conference was held in Ede, this years topic was System Administration in general.. which means there was a pretty wide range of talks ... some of the talks were extremely interesting and gathered a lot of people , others really shouldn't have been put in the main room.. Frequenly organising them myselve it's always a difficult choice for a conference organiser.

Must say this was one of my better talks .. it all went smooth and nicely fit within time. Probably the promise of drinks and food after my talk helped some.

I started it of by showing the audience Patrick's opening Devopsdays'09 Video ...

I slightly modified the the slides for my NLUUG presentation, but they are based on the talks I gave on the same topic before

De v ministration View more presentations from Kris Buytaert.

Please note that the Devops definition I give early in the
slides is there to misguide the audience ... :) Everybody knows drinking beer and eating sushi is just a start in the journey when you want to become a #Devops :)

The fun part about conferences often are the speakers dinners, you get to sit down with interesting people and talk about a variety of topics such as panacotta recipes and configuration management ..

It was fun ... too bad it took so long to drive there.. good thing we got back pretty quick..

Technorati Tags: conference devops nluug presentations Share with Shareomatic! --> Trackback URL for this post: http://www.krisbuytaert.be/blog/trackback/1003

You might have read about it .. al over the internets. but today a big step for the implementation of global DnsSec implementation is being made .

You might want to read up about the impact.

And test here if you are unsure about your situation.

Technorati Tags: dnsproble dnssec eiafdp Share with Shareomatic! --> Trackback URL for this post: http://www.krisbuytaert.be/blog/trackback/1002

For those who've seen my presentation on MySQL HA, you already know that I often use a multimaster setup with a meta OCF resource that groups my favoured MySQL instance with the service ip , using a meta resource means that pacemaker monitors mysql, but it doesn't actually manage it. It's an approach that works for us.

One of the other approaches I will be looking at soon is the freshly released OCF resource that Florian announced last week.

Back in the days our approach meant we didn't have to use clone resources, which you might remember being pretty buggy in the v2 era, not wanting to use clons resources isn't really a valid reason anymore these days . I've also frequently mentioned the combination of using DRBD and MultiMaster replication, using this set of OCF resource makes that a lot more easy ..

Now all I need to do is find me some time to validate this setup.

Technorati Tags: cluster ha mysql Share with Shareomatic! --> Trackback URL for this post: http://www.krisbuytaert.be/blog/trackback/1001

Whenever I give my Cloud security talk there's a slide in there talking about the most scary idea about Cloud and Security, the fact that Marketing people will build things on their own while IT, or any other departement isn't involved, and as we all know marketing people have no clue about security, it's not on their mind they won't even think about adding some sort of security to their application.

So IT isn't involved, Development isn't involved , and Operations isn't involved ...

Ages ago.. well.. about a decade I was working in those very marketing departments sitting there, writing code, hired by the marketeers, not by IT , the marketing PM did the talking to IT , we still had to go trough their IT department to get stuff deployed.

The marketing people had to deal with their impossible deadlines, a nationwide tv or radio campaign that was going to be launched , with a supporting website which meant that the website functionality needed to go live just before the first airing of the commercial. Obviously the website was lower priority than finding a famous voice or face to record the commercial with, so it became only late in the planning.. even more obvious was the fact that talking to IT about getting these new features deployed was even later on their planning .

Back then, part of my job was to smooth that process, my role was both creating the technical backend for the sites , putting them in production and doing the daily maintenance afterwards ...

Looking back at those days I realize the pains of both deployment and procurement, getting a new machine racked and then installed up to a bare os installations took up to 6 weeks, in a marketing driven world that meant that I'd often had to bypass the whole procurement process of expensive sunboxen and had to quickly deploy a linux box under my desk that could be used to move to production as plan B , and trust me .. we had to use plan B a lot ..

Letting nontechnical people deploy stuff in the cloud will only widen the gap, but getting involved early enough in the concept fase of a project with a good devops methodology/team in place will give the business people the opportunity to learn that things have changed , it doesn't take 6 weeks anymore to get an expensive Sun box racked and a Solaris instance installed after which a team of engineers needs to install an application server, then a different team needs to install the database etc .. these days it's a virtual machine instantiation and a couple of recipes ,in that way we can get manageable, reproducible and scalable deployments in no time.

Technorati Tags: cloud devops history marketing opensource Share with Shareomatic! --> Trackback URL for this post: http://www.krisbuytaert.be/blog/trackback/1000