Openembedded and Bitbake Stupidity and Odd results from Openembedded
By Bill » Sunday, January 17 2010, 17:10
If I live to be 100, I'll never understand why Linux programmers think the way they do...
Since obtaining my Micro2440 board, I've been using OpenEmbedded to create kernels and root filesystems for u-boot. I don't understand why there is a need for 5 million tiny files called recipes. I mean, I understand what they are trying to do, but having to load almost 8000 tiny files each time you go to build something is incredibly STUPID. It's the same with Portage in Gentoo. Zillions of tiny little files that have to be synced up when you update portage. It's crazy retard thinking. I personally don't like the idea of having millions of files like that on the system, but if I were to do something like that, I'd build a master index of them basically compiling them in to a large database that can be easily accessed and then when it needs updating, you run a program to rebuild that index.
For what I do, I don't have a need to rebuild that stupid cache and read all those files in each time I want to compile a single thing in to an ipkg file. I'm not modifying the the .bb recipes so theres no need. People who do that a lot, well they should have to manually rebuild a cache or index each time.
Bitbake is crap, however... So far Openembedded and Bitbake are the only things that have produced runnable images for me. My attempts to use Buildroot have so far failed with strange compiling errors and missing files. Now Openembedded is missing a lot of things too. Try building maemo, or a maemo image. Go ahead. The recipes are there, so you assume it will work, right? Nope. Missing packages. And nobody at all on the net seems to talk about it or care. Try building the Gnome task. More missing packages.
Without going off on a total rant, I just want to make it known that I'm not a total newbie to Linux. I've been using Linux since the early 90s. I've used it heavily in my business reselling servers and setting up web hosting. So I've been around the block when it comes to Linux. And it's been my experience over the years that most things that have anything to do with Linux are half done, mostly broken, and largely disappointing. You get your hopes up about something just to have a bunch of dimwitted teenage programmers who seem to run the Linux community, crush your hopes. And not being a C programmer myself, I can't fix or even contribute. It pains me!
I hope to soon write some documentation on here about Openembedded but currently I'm in a testing phase with it. I'm having some odd results. I can build a base console image and flash it and it works. But I can build the same image 2 or 3 times and try to flash it, and each time it'll either work or it wont. The problem centers around the root filesystem not being recognized. I get errors about Magic something or other not found at the right spot. I thought it was my eraseblock size and I corrected that (Which I will later document) in the openembedded code for the mini2440 since it's assuming we all have 512byte page size and 16k eraseblock (sector size as reported by u-boot). But it's still randomly working. My current attempt was to ditch 64bit Debian and move to i386 to see if that makes a difference. I know that GCC can be extremely fragile. But I'm also investigating the possibility that U-Boot could be the culprit as I'm also having trouble with kernels. Some kernels will work, some are being reported as Invalid format for bootm.
I'll be posting more here soon about my results.

Comments
What a rant, it sounds like you haven't read the manual!
If you want to bitbake a single recipe all you have to do is:
bitbake -b /bbfilepath/recipe.bb
And obviously you didn't read my blog entry. I was complaining about the loading time of all the recipes.... EVEN WHEN YOU BITBAKE ONE SINGLE RECIPE. Also, just in case you weren't aware, I *am* actually new to this. Which I've said quite a few times on this blog, had you bothered to read anything else. I actually spent a month before I bought the Micro2440, doing nothing but reading and learning. But reading and actually doing are different things. Also what a manual sometimes says will happen, isn't always what happens in real life. I'd think you'd know that. ;) Anyway, my rant was because I was frustrated. This blog, if you've read any more of it, is about my learning experience.. the good and the bad...
Hello,
I've been trying to build u-boot and a file system using bitbake with no success. I have the 128M micro2440 and bitbake does not seems to run at all. it either complains about setting " the 'PRESISTENT_DIR' or 'CACHE' variable" or it generates an error that says "AttributeError: BBCooker instance missing attribute 'cook'.
can you please help.
I've follow your instructions under " Getting started with OpenEmbedded"
but still can not seem to pass these two errors.
Thanks in Advance
Are you positive you've exported the environment variable BBPATH to point to the main directory where OPenEmbedded is? You must do that each time you open a shell and begin working on it. Or you have to put it in your .bashrc file in your home directory. Please let me know if that helped. If not, we'll figure it out :)
Hi Bill ,
thanks for the reply. I managed to get bitbake to work. I was able to build a uImage-micro2440.bin . However, I am not sure if everything went well . I got uImage but no u-boot or JFFs2 file system.
Does bitbake generate the bootloader(uboot) and a file system( jffs2) or is it a tool for creating kernel images only...?
Thx.
What recipe did you bitbake, Mfofana?
"I was complaining about the loading time of all the recipes.... EVEN WHEN YOU BITBAKE ONE SINGLE RECIPE."
Which is why i pointed out you can use the -b switch to avoid loading all the recipes!!!
Well, Doug, you're right that it's supposed to do that but mine isn't. Not even with -b... I'll try again when I'm back to debian. I've been testing some things in gentoo.
Bill, I completely understand your pain. I also have been using Linux ( among many other OSes ) in one form or another since the 90's, and have experienced much ( if not all ) of what you talk about here. I too am very disappointed with many aspects of the OSS movement as it were. Going by what you say; For the same reasons. At the same time , I am far beyond what people consider a power user, and Ubuntu is not on my very short list of acceptable distro's. This does not mean I consider myself to know *everything*. It just means I have the patience, and ability to learn where computer systems are concerned ( plus having experience with computers back as far as 81 does not hurt either ).
With the above said. I would like to point out that you ( not unlike myself probably ) are using OpenEmbedded for specific purpose(s). Getting side tracked because of issues in the build process would be a source of frustration for just about anyone. You should take a break, and perhaps read more. Then perhaps, your problem will come out, and you will be better equipped to deal with it. This is not to say that I disagree with most/all of what you say ( because I really do not ). It just means that your end goal is not being achieved, and that does not help you at all.
We all also need to understand that the people who work on projects like this make no money from it. So, we get what we pay for . . . We would all like it to work for us, but if it does not, can we really blame them ? I think not.
I would also like to point out that when I do something as complicated as this. I find that taking notes on what I do, and when helps me greatly.
I use notepad++ , and just write down the steps I do when I do them. That way, when something goes wrong. I have a better chance of finding the problem. If no problems at all. Well then, I have just written myself a howto, or a recipe for success. ; )
Hi William,
Well the old saying of you get what you pay for is very understanding but Linux and OSS people want their products to not only be "hobby" projects, they want to push Linux and other open source solutions to business levels. Thats where I have a problem. A lot of people claim it's ready for that and in some senses it is, but in others it's not. The Open source idea is really good, but it's really not super easy to organize and manage large amounts of people who work on projects and get them collaborating efficiently. (At least in my opinion). And thats what leads to broken packages, and what not.
You said that I should read more. You can only do so much reading because theres only so much actual documentation on the products. If you read the openembedded manual, you're still left with a lot of unanswered questions. So you go to the forums and wikis of the web to look for answers. You usually can find what you need if you look hard enough but not always. Especially when the software (such as openembedded and its recipe maintainers)behaves differently than what the manual states, and with each version it works or doesnt.
You're right that it's frustrating ;) I wrote my original post here because I really needed to vent. But if you read the rest of the blog, I did overcome a lot of the obstacles in my early processes of learning this stuff.
Notepad is a great way to keep notes. Thats what I was doing in the beginning. I converted my notes over to this blog because I thought people might want to see what I was learning as a beginner because most of the information out there assumes you know everything already. ;) Thanks for writing! It's great to see a fellow old-timer ;)
Bill, well when I said to read, I meant in general. Read the docs, read something on programming C. Hell, even read The Hobbit.
The idea of course, is to step back away from something that is frustrating; To keep from making things worse.
I would suggest that you forget about running debian x64. I have no doubt it would be no end of frustration for you. Do not get me wrong though; debian in my mind is the *only* way to go, and I would like that their 64-bit version were 100% stable/reliable. But this is not what I have been reading from different people. Well, I imagine it could be with the right( specific ) hardware.
Also, instead of switching back and forth between debian and gentoo. Why do you not consider using a Virtual machine? Xen would be a bit over complicated for this situation. But perhaps if you run X, you could use something such as virtual box.
Try not to laugh when I say this. But I personally run Windows 7 enterprise( Ultimate ) x64 as a host for virtualbox + debian. It is my hope that once I do get a working oe image. That I will be able to develop( I do program a bit ), and test it as a virtual machine first. I have lots of ideas that I want to work on. This should help facilitate what I want to do. Anyways, it is just a thought.
Hey William,
I've been kind of on a break lately. I got two new quad core systems in that I've been focusing on right now. So the Mini and all the work on that has been suspended until I'm done working with that and getting those setup. But you're right about needing a break. I tend to do things like this because under normal circumstances, I have very little to do during the day. I need stuff to keep me challenged. But that challenge only goes so far before I become frustrated and cranky. Thats where this original post came from. I'm sure it won't be the last post of it's kind. I've been blogging since before blogs were even popular. My style is typically a rant. This is my first attempt to write a technical help blog... so occasionally a rant might slip in. It's my style :)
I've been using Debian 64 quite a bit. My only problem with it is its older packages. Debian has always been that since years and years ago when I used it as my main server for my business. When I ventured out and tried other distros, I realized others were much more up to date. But still, for me anyway, Debian is always very very stable and according to everything I've read has all the support for this embedded stuff. The 64bit version is much faster on my Laptop than the i386 version. You can see the difference in just how the kernel boots. Compiling is slower as well. So thats why I really insist on 64bit if all possible. Try it again sometime :)
Xen was a big part of my hosting business several years ago when it was new. I loved it. But as you said, it is over complicated for this. I've considered VMWare and I had been trying to get QEMU working but QEMU seems to be too big of a hassle. Over complicated in how you handle the virtual disk. But virtualization is definitately an option now. I have these 2 quad core systems I'm itching to compile on to test the speeds.
I'm a BIG Windows supporter. I couldnt function without it. Linux just doesnt make me happy in so many ways. So on my own personal machines I always run windows. I always have. And Windows 7 is the absolute best version of Windows to ever come out as far as I'm concerned. It's blazing fast. Never feel weird around me about talking about Windows :) I love it ! Hehe
Well, I feel the same way about Windows. In a lot of ways, I feel Linux should be more like windows, and I also feel that Windows should be more like Linux in many ways as well. It has been my experiences though, that Linux users can be zealots, and can not see passed their own nose as they're looking down it at Windows.
Anyways, with the above said. It is this users opinion that Windows has *nothing* on Linux when it comes to embedded devices. Not because of polish, not because of support, but because of flexibility/freedom. Microsoft still has a lot of things to learn in this area the way I see it. Of course, they do, what they do to protect their I.P. Also, I am fully aware of Windows CE ( CE builder, and the PE environment ), as well as the implementations of Windows embedded that they have had for several years now.
Awe, is the word I would use to describe my feelings when thinking of the Linux initial ram disk. Simple, and very flexible at the same time. So many things could be done at boot time it just boggles the mind. But something as "simple" as booting a copy of Windows from its own software raid array is simply unheard of. Not true with Linux, and that is just the beginning of our options.
However, my point here is not to advocate one operating system over the other. I use, and like them both. For different reasons. Truthfully, I am not sure what my point is here other than this is how I feel about Linux, and Windows.
Now, as for Xen. It has been my personal experiences that it would not play well with anything as complicated as this. Xen on its own is a complicated beast, and add something like Openembedded into the mix, and you're nearly guaranteed a recipe for failure. A year, or two ago, I attempted to get Xen, and iSCSI working together, and it was a complete bust. I am not saying it is not possible, but it would have been a huge hassle assuming it could have worked ( at that time ). Around this time is when I also formed my opinion concerning Ubuntu. I was doing tons of testing with iSCSI, AoE, NFS, and every other network storage protocol you could think of. Ubuntu was constantly crashing ( and I mean; bringing the whole system down! )when using XFS with Samba, and just about every other protocol. While the same box ( removable cartridge hard drives ) Running Windows XP with an implementation of iSCSI, or standard network shares was rock solid stable. Yeah . . . I know . . . Imagine my surprise.
Anyways, this is part of the reason I chose to delve into embedded systems. I want to eventually make a very low power network storage system. We here ( a buddy of mine, and myself ) are totally off grid ( mostly solar/wind power, plus backup generators ). And then there are other projects I am wanting to do. Too many to list, without actually needing to write my own blog ( heh ). My buddy has been an electronics engineer for over 30 years, and I have been programming for around 15 as a hobby. Between the two os us, we should be able to get something working. Yet, at the same time. I still feel like an utter "newb" when it comes to various aspect of embedded system. And I have been reading for more than a year now !
SO now, I have rambled on long enough. I have been considering building my own technology site, and if I do you can bet it will contain my random thoughts on various aspects of embedded systems designs. If, and when that happens, I will make sure I shoot you a link ( if you're interested ). One thing I am thinking though. I may try something first with an x86 platform first. As I already am up to my neck, and I do not want to be complete buried alive just yet ; )