Alex Scoble's IT Notes - An Information Technology Blog

Networking for OpenStack

When using Mirantis Fuel Web to build up an OpenStack cloud, it requires 6 separate networks to be created. Simplest way to do this is by using virtual networks (VLANs), however this presents a few challenges because of the complexities of configuring VLANs.

This post is specific to Cisco switches, but may be relevant for other networking equipment as well.

For test purposes, this was all done on one switch and each bare metal server had one interface cabled to the switch. In production environments, it is likely that, at a minimum, a completely separate storage network would also exist, but more complicated networking schemes are quite possible.

For each port on the switch that will be connected to one of the bare metal servers, the following needs to be done:

switchport mode trunk
switchport trunk allowed vlan 10,100-104
switchport trunk native vlan 10
spanning-tree portfast trunk
switchport nonegotiate

Trunking mode must be used because each interface needs to be able to handle all 6 of the virtual networks. In the normal access mode, an interface can only communicate over a single VLAN.

For each interface, it's a good practice to specify which vlans it is allowed to handle.

Trunk native tells the switch that untagged traffic goes over this vlan. VLAN tags are the mechanism that allows a network interface to communicate over multiple discrete subnets. So in this case, only traffic going over vlans 100 through 104 will be tagged.

Spanning-tree portfast trunk tells the switch that this port is a trunk connected to a server or PC and to bypass the usual network loop checks.

Switchport nonegotiate forces the switch to treat this interface as a trunk because the connected device does not understand the protocol the switch uses to negotiate trunking vs access modes.

In the switch config the VLANs also need to be activated using:

vlan10,100,101,102,103,104

For those who normally work with vlans in switchport access mode, the switch automatically does this, but apparently, when working with trunks, adding a vlan to a port does not automatically create and enable that vlan.

I am not a networking expert so some particulars here may have been missed, but I hope this is helpful.

Microsoft's "Innovative" New UI

I'm not a big fan of Microsoft's new UI that they created for WinMo 7 and Windows 8. It feels unfinished and unpolished like a cheap piece of IKEA furniture. I never thought that any developers would find it innovative enough to copy it, but that's exactly what the devs at Slacker did for their new UI. In terms of usability, the UI is good, but the blue squares thing just doesn't look good at all when compared to stuff like Android or iOS.

Restoring a MySQL database from files

If you are a relative MySQL newbie like me, and someone tells you to backup your databases, you might make the same mistake that I did and copy the database files (all of those .frm, MYD and .MYT files) to a backup folder instead of using MySQL's built in backup method.

Of course, when you make this kind of mistake, often you are required to restore from backups, so you copy the files back to their original location and MySQL bombs on you when you try to start it.

In my case, I backed up the entire /var/lib/mysql directory to /tmp/mysql-backups. When I copied the /tmp/mysql-backups stuff back to /var/lib/mysql, nothing worked. Doing a 'cat /var/log/mysqld.log' showed a "Can't find file: './mysql/plugin.frm' (errno: 13)" error.

Turns out that you have to reset permissions on those files and do some work with SELinux to get that to allow the mysql account to access the databases.

This help is specific to MySQL using InnoDB on CentOS 6.x. I have no way of knowing if it will work for any other configurations.

Here's how to restore your databases and reset the permissions assuming that you have a backup of the entire /var/lib/mysql directory. I'm going to do a bunch of similar chown/chmod statements below just to be sure. You could do the same thing by doing
'chown -R mysql:mysql <directory>' and you'll want to do the separate commands for chmod since the directories and files should have different allowed operations.

rm -rf /var/lib/mysql
mkdir /var/lib/mysql
cp -r /tmp/mysql-backups/* /var/lib/mysql
chown mysql:mysql /var/lib/mysql
chown mysql:mysql /var/lib/mysql/*
chmod 660 /var/lib/mysql/*
chown mysql:mysql /var/lib/mysql/mysql
chmod 700 /var/lib/mysql/mysql
chown mysql:mysql /var/lib/mysql/mysql/*
chmod 660 /var/lib/mysql/mysql/*

Do the chown and chmod commands as shown for any other directories that you have under /var/lib/mysql

If you tried to start mysql right now, you'd still get the same error though if you have SELinux turned on, so now we have to fix that.

chcon -v --type=mysqld_db_t /var/lib/mysql
chcon -v --type=mysqld_db_t /var/lib/mysql/*
chcon -v --type=mysqld_db_t /var/lib/mysql/mysql
chcon -v --type=mysqld_db_t /var/lib/mysql/mysql/*

And so forth for any directories that you have under /var/lib/mysql

If you started mysql now, you'd see this error now "[ERROR] Can't start server : Bind on unix socket: Permission denied". SELinux is still blocking access so now you have to do the following:

semanage fcontext -a -t mysqld_db_t "/var/lib/mysql(/.*)?"
restorecon -Rv /var/lib/mysql

It is likely that you will only have to do the 'restorecon' command above, but I'm being thorough here.

You can now start the mysql service ('service mysqld start') and it should work without issues.

Enterprise Linux tools need packaging

With few exceptions (Puppet being one of them, because it just rocks) open source enterprise tools need to be packaged and available in one of the mainstream repositories (even EPEL is good enough! The bar isn't that high!) or your tool just isn't ready for primetime.

Looking at you Plone, LogAnalyzer, nxLog and LogStash! OSSEC gets a pass, but just barely because they are using the funky Atomic repo.

This is where tools like Zabbix and OSSIM come out ahead. The last thing you want to be doing when managing an enterprise, particularly in the industrial control space because of the funky requirements, is worrying about manually updating ANYTHING.

Really, the only reason why Puppet gets a pass is because it's so hugely useful in an enterprise.

And no, I don't want a bunch of excuses on why your tool isn't in a repository, whether it's for Debian, Ubuntu or the Red Hat side of things. If it's not available as a managed package, chances are your tool isn't going to be deployed by me or other IT people like me...and we tend to be a very stodgy group.

Cheat sheets coming out our sockets and ports!

Need a cheat sheet? Click here to go to a blog that has listed tons of them for you.

Covering everything from MS Office to AS400.

Dell no longer selling HP + Exchange admin tool found

I've posted two more items on my ComputerWorld blog.

Click here to get to the first item where I talk about how Dell is no longer selling any HP branded products.

Use this link to go to post about Exchange tool that I found to quickly change all permissions of inbox subfolders of a specific mailbox.

Quoted In ComputerWorld Article On IT Blogging

Heh, nice to see that my brother, Robert, isn't the only one getting quoted in the media.

Although it isn't Fortune or Forbes, I was quoted yesterday by ComputerWorld in an article on IT vendors and managers blogging. Although the article is mainly about how vendors are embracing blogging while IT managers/customers are mainly not taking the plunge, also uses quotes from me on why IT managers should quote.

Good article, but I would have liked Patrick Thibodeau to link to my blog and to my firm's website. Can't have everything though, so I'll just be happy with the fact that I've now been quoted by an actual news outlet.

Wrong Name Update Update (not a typo)

Adam Gaffin, the person who made the original post actually checks his comments. He saw my comment and fixed my name. Thanks. :)

They Linked To Me But Got The Wrong Name

Just saw that Network Fusion is linking to me here, but what's funny is that they say that someone named Scott Mace is dealing with the issue and not me.

Hopefully someone sees that and comes up with an answer.

Reminiscing on Internet Technology

It was about 10 years ago that I first heard about the World Wide Web from my brother Robert. I remember him going on about these things called web pages and how it was all very cool. I didn't have a clue what he was talking about and just shrugged it off. Needless to say that Robert is on the bleeding edge of software technology, which is probably why he is where he is today (that and his "geek" personal skills).

But he was right about the technology of the Internet that far back, it was cool and while it may not have revolutionized the world, it sure did change how we shop, work and interact with other people.

I have no idea how long he's been working with RSS, but it's been at least a year, maybe two or even longer. But within the last 18 months or so is when he really started to blog about how RSS and apps like NewsGator were changing the way he interacted with the web. As usual, I was like "yeah, yeah, whatever" and went on, until last Friday when I downloaded FeedReader (ya gotta love free apps, but that's a topic for another day). Already I can see how much better it is to read my favorite sites this way.

It's akin to how enterprise server products (like Symantec AntiVirus CE) changed the way we manage things piecemeal to administering and monitoring everything from a central location.

And already I am wishing that all web sites had RSS feeds, but unfortunately some of my favorites like Overclockers.com and Roger Ebert's website don't have a feed.

Anyhow, the point of all this rambling is how cool it is how far we've come in 10 years on the net. Still have a long way to go though on what is possible both technologically and societally.

I just wonder what my brother is going to find next that will capture my attention two years afterwards. Might have to wait another 10 years to find out, but it's out there somewhere.