Sign in with
Sign up | Sign in
Your question

Creating Specialized, Custom Linux Distributions

Last response: in Linux/Free BSD
Share
a b 5 Linux
September 13, 2011 4:08:20 AM

Hey everybody,

I've recently been considering taking on a project that is way out of my comfort zone. :D  Basically, I'm wanting to learn to put together custom Linux distributions for specialized applications. I have a few ideas at this point, but nothing is set in stone. I'm basically just wanting to learn how to do this for my own self satisfaction, and to expand my current knowledge of Linux/Unix in general. Also being back in school, I'm just irking to find some programming projects :D 

What I'm looking for are some resources on how to go about doing this. Most of the results I've found are just LFS, or guides on modifying current installations. I can't really find much on starting from scratch. I just honestly have no idea where to start.

Any suggestions, links, etc. are very greatly appreciated. I look forward to discussing the feasibility of this all with you guys :) 

Thanks,


- Jesse
a b 5 Linux
September 13, 2011 6:20:02 AM

It would be quite hard to start from scratch. Have you considered a spin on a distro?
m
0
l
a b 5 Linux
September 13, 2011 2:43:27 PM

amdfangirl said:
It would be quite hard to start from scratch. Have you considered a spin on a distro?

+1

If you wanted to continue down this path, you'd either a) need to purloin one of the "install choices" programs and re-purpose it for your needs b) write your own or c) force a default install and make users change things if needed.

I personally feel just going through a LFS build will essentially give you a deep understanding of what goes into a linux system. Gentoo also gives you this experience but has a nice little package management system called portage (base don FreeBSD's ports system but some nice improvements)
m
0
l
Related resources
a b 5 Linux
September 13, 2011 5:50:13 PM

Alright, thanks guys. As I said, I wasn't sure how feasible this was to begin with. I'll just have to get into some HEAVY modification :D  I'll also run through LFS and Gentoo as time permits.

Thanks again,


- Jesse
m
0
l
a b 5 Linux
September 14, 2011 1:58:44 AM



Working on that as we speak :D 

(Homework can wait, right?)

EDIT: Also, I was wanting to work on a distribution for embedded solutions. Any particular topics I need to research, or is it just the basic stuff like optimization and low-resource programs/packages?
m
0
l
a b 5 Linux
September 14, 2011 3:20:01 AM

Pyroflea said:
...EDIT: Also, I was wanting to work on a distribution for embedded solutions. Any particular topics I need to research, or is it just the basic stuff like optimization and low-resource programs/packages?


Look into:
Yocto - a build system to create system images
OpenEmbedded - also abuild system, Yocto is based on it
CodeSourcery Lite - An easy-to-use, free-as-in-beer (and eventually makes it back to the gcc project) cross toolchain
Angstrom - a base distro used by the first two
busybox - A tool that replaces many of the standard GNU utilities on resource-limited devices, may need to configure and build your own to fit needs
u-boot - Embedded bootloader du jour. Useful as you may need to configure how the device boots or use it for lowlevel HW debugging
m
0
l
a b 5 Linux
September 14, 2011 3:30:58 AM

Thank you kindly :) 
m
0
l
September 14, 2011 7:37:43 PM


Excellent guide.
Consider Arch Linux, although it is a bit more than what you want. It comes with a package manager and basic CLI applications. There is no GUI on it, and really just a CLI waiting for you to customize it.
http://archlinux.org
m
0
l
a b 5 Linux
September 14, 2011 10:27:51 PM

Corrytonapple said:
Excellent guide.
Consider Arch Linux, although it is a bit more than what you want. It comes with a package manager and basic CLI applications. There is no GUI on it, and really just a CLI waiting for you to customize it.
http://archlinux.org


Thank you for you suggestion! I have in fact used Arch, and if I did end up going the modification route was debating between Debian and Arch :)  I just have an interest at an even lower level than Arch offers, and was wanting to get into modifying/compiling my own kernel even. We'll see how things go though :) 
m
0
l
a b 5 Linux
September 15, 2011 12:59:17 AM

Lol, PM Linux_0.

Maybe you might get a responese.
m
0
l
a b 5 Linux
September 15, 2011 1:00:03 AM

Pyroflea said:
... I just have an interest at an even lower level than Arch offers, and was wanting to get into modifying/compiling my own kernel even. We'll see how things go though :) 

Gentoo.
m
0
l
a b 5 Linux
September 15, 2011 1:19:26 AM

amdfangirl said:
Lol, PM Linux_0.

Maybe you might get a responese.


Normally I would, but he seems to be MIA as of late. :lol: 

bmouring said:
Gentoo.


Gentoo it is then :D  I'm hoping to have time to try this all out on the weekend. I had a couple assignments dumped on me, but they shouldn't take too long ;) 
m
0
l
a b 5 Linux
September 15, 2011 6:16:00 AM

If you're lucky I might design some spiffy icons for you.

Depends how bored I am.
m
0
l
a b 5 Linux
September 15, 2011 6:58:29 AM

Pyroflea said:
Normally I would, but he seems to be MIA as of late. :lol: 

Ah, linux_0, we miss you. In all fairness, I guess I went MIA from '07 until recently.
m
0
l
a b 5 Linux
September 15, 2011 4:46:01 PM

amdfangirl said:
If you're lucky I might design some spiffy icons for you.

Depends how bored I am.


I'll be sure to contact you once I get to that point. I'm not the artsiest person out there :D 

bmouring said:
Ah, linux_0, we miss you. In all fairness, I guess I went MIA from '07 until recently.


That is true, welcome back (although it's a tad belated ;)  )
m
0
l
a b 5 Linux
September 15, 2011 11:59:27 PM

Got some more questions! Seems the more I learn, the less I know :D 

I also got recommended some other tools on another forum (mostly Scratchbox and Buildroot), so I've got some comparing and whatnot to do.

Another thing I've run into is QEMU. I've found guides for compiling ARM kernels and running them through QEMU on an x86 PC for testing/dev purposes (at least I think that's what they meant :D ). Would this be the way to go just as far as learning/developing things, or would there be a better option? (Don't have the ARM hardware yet, waiting for the release of a certain board later this year)

Also slightly less related, but due to the security breach of the kernel.org site, I can't seem to find a download for the kernel itself (following a guide for something :p ) anywhere else. Anybody know where I can find it?

Thanks,


- Jesse

EDIT: apt-get source linux-source-2.6.38 :D  That'll do for now, just for practice
m
0
l
a b 5 Linux
September 16, 2011 2:42:18 AM

Pyroflea said:
Got some more questions! Seems the more I learn, the less I know :D 

I also got recommended some other tools on another forum (mostly Scratchbox and Buildroot), so I've got some comparing and whatnot to do.

Another thing I've run into is QEMU. I've found guides for compiling ARM kernels and running them through QEMU on an x86 PC for testing/dev purposes (at least I think that's what they meant :D ). Would this be the way to go just as far as learning/developing things, or would there be a better option? (Don't have the ARM hardware yet, waiting for the release of a certain board later this year)

Rasberry Pi?

Scratchbox is a toolset for cross-compiling, usually single applications. Building components for my phone (n900) uses scratchbox, and it seems pretty easy to use as far as cross-compiling within the confines of the target system go (making sure it can use the libraries that will be available, etc.)

I haven't used buildroot before, but it seems fairly reasonable, so long as your hardware's supported (or you're willing to invest the time to make it supported)

I've heard QEMU is a fine way to approach testing that your applications work properly on an arm system, but usually pays off a bit better if you can get ahold of an already-created image that's fairly close to what you expect to see on the device
Pyroflea said:
Also slightly less related, but due to the security breach of the kernel.org site, I can't seem to find a download for the kernel itself (following a guide for something :p ) anywhere else. Anybody know where I can find it?

Thanks,


- Jesse

EDIT: apt-get source linux-source-2.6.38 :D  That'll do for now, just for practice

https://github.com/torvalds/linux, how to use git
m
0
l
a b 5 Linux
September 16, 2011 2:57:49 AM

bmouring said:
Rasberry Pi?


How did you know! :D  But yes. The B Model of course.

bmouring said:
Scratchbox is a toolset for cross-compiling, usually single applications. Building components for my phone (n900) uses scratchbox, and it seems pretty easy to use as far as cross-compiling within the confines of the target system go (making sure it can use the libraries that will be available, etc.)

I haven't used buildroot before, but it seems fairly reasonable, so long as your hardware's supported (or you're willing to invest the time to make it supported)

I've heard QEMU is a fine way to approach testing that your applications work properly on an arm system, but usually pays off a bit better if you can get ahold of an already-created image that's fairly close to what you expect to see on the device


Again, they were just recommended to me and I still need to do some research. I'm still working on figuring out exactly what everything you recommended me is. :p 



Thank you! I did a search there last night, but I must have just screwed something up royally :D  Thanks for pointing it out though.
m
0
l
a b 5 Linux
September 16, 2011 3:57:05 PM

I think for now I'm just going to tear apart Debian and work with that. I plan on removing literally everything not needed (this distro will JUST be for this board), and then adding in whatever drivers required, packages I want, etc.

I still plan on going through with a Gentoo install this weekend (Stage 2 I'm thinking), but that will just be for the purpose of learning.

Thanks,


- Jesse
m
0
l
a b 5 Linux
September 16, 2011 4:18:39 PM

Stage 2 vs. Stage 3 will only win you lower compile times if you plan on creating an x86 (32-bit) install on a more recent processor at the cost of additional time spent upfront. You literally only run a few scripts as I recall (and unless you plan on reading the contents of those scripts, doesn't teach you too much about what's going on). x86_64 predefines a lot of functionality/capabilities such that it's difficult to really squeeze any extra performance out of the build system.

In short: unless you really feel it will add something, go with Stage 3 over Stage 2
m
0
l
a b 5 Linux
September 16, 2011 4:53:05 PM

Alright. I must have a misunderstanding of what the difference is. I'll read up on the differences and then start fooling around. I basically have only gotten around to partitioning the drive so far :p 
m
0
l
a b 5 Linux
September 16, 2011 9:04:50 PM

I've got the arm version of Debian running in QEMU. Woo-hoo! Time to start modifying :p 
m
0
l
a b 5 Linux
September 17, 2011 12:30:33 AM

Pyroflea said:
I've got the arm version of Debian running in QEMU. Woo-hoo! Time to start modifying :p 

Good on ya! Happy hacking:)  let us know how it goes
m
0
l
a b 5 Linux
September 17, 2011 12:33:21 AM

Pyroflea said:
...I basically have only gotten around to partitioning the drive so far :p 

Experience here alone I've used more times that I care to count (including my recent work at my real job; a customized Linux distro running on (you guessed it) recent ARM platforms; those flash devices don't partition themselves... yet)
m
0
l
a b 5 Linux
September 17, 2011 12:58:42 AM

So basically upon doing some more research, I will need to do less work that I originally thought to get it to simply work. However, to get it to work how I want it to will be a lot more work :D 

Basically their demo was running Debian + LXDE. Neato, we know it's powerful enough to do what I wanted. What I want to do is just streamline my installation as much as possible (it's still between basing it off Debian and Arch), and remove EVERYTHING that isn't required. I wouldn't mind doing some tweaking for various other things as well.

From what I've gathered, the trickiest part will just be getting it to boot properly. [According to my current understanding], some code on the ROM directs the board to boot off of a FAT32 partition on the SD card, which holds a module that they have developed. From there, it will load a boot loader on another EXT2/3 partition, and we go from there.

Slowly figuring things out :D  I'm mostly just going to read up on modifying things to suit my needs at this point, as well as cross-compiling to support ARM.

Thanks,


- Jesse
m
0
l
a b 5 Linux
September 17, 2011 4:29:23 AM

Yep, that sounds familiar, for TI OMAP boards it's
ROM->(FAT32)->MLO/x-loader->u-boot->kernel
other implementations may vary slightly
m
0
l
a b 5 Linux
September 17, 2011 5:31:30 AM

Yeah I assumed it was pretty standard. Unfortunately most of the work I need to do can't be done until the board is in my hands, so I'm limited as to what I can do at this point. Oh well.
m
0
l
a b 5 Linux
September 17, 2011 7:31:53 AM

Write a blog about your project?

Attract some Linux experts...

Mmm... had a dream Linux_0 made a custom spin of Linux for me which had a really cool interface, like GNOME 3 but tabbed and every programme I like was integrated into the 1 programme with all the Adobe CS tools as well.

That was a cool dream.
m
0
l
a b 5 Linux
September 17, 2011 5:14:57 PM

amdfangirl said:
Write a blog about your project?

Attract some Linux experts...


I may just do that ;) 

amdfangirl said:
Mmm... had a dream Linux_0 made a custom spin of Linux for me which had a really cool interface, like GNOME 3 but tabbed and every programme I like was integrated into the 1 programme with all the Adobe CS tools as well.

That was a cool dream.


LOL, no comment :D 
m
0
l
a b 5 Linux
September 19, 2011 2:09:34 PM

bmouring said:
Ah, linux_0, we miss you. In all fairness, I guess I went MIA from '07 until recently.


I just assumed it was a drawn out toaster upgrade ;) 

A quick note on the RPi. It's boot loader is a little strange in that it actually uses the GPU for initial boot before resetting and passing control back to the main CPU. There was a little discussion about it on the forums but it RPi is your intended target then take a good look at the HW before you start shaping your solution to much.
m
0
l
a b 5 Linux
September 19, 2011 2:15:41 PM

amdfangirl said:
.. all the Adobe CS tools as well.

That was a cool dream.


That you were able to visualise Linux_0 working with closed source shows the volatility of that overly creative imagination of yours... Mind you it's probably a safer mental image than him becoming your custom GIMP master :o 
m
0
l
a b 5 Linux
September 19, 2011 2:48:11 PM

Oh no.

You get me wrong.

He made a fully open-source alternative with more features than the Adobe Suite.

It just resembled the CS interface. I think.

Weird dream.

I guess this rules out me telling you all about the second part of the dream that involved Linux_0, open souce pistols and katanas, CERN, the GDI Ion cannon network, scientfic linux, lightsabres, the Microsoft Windows-Apple OS X "icloud", blue terminal and the red command line.
m
0
l
a b 5 Linux
September 19, 2011 3:15:29 PM

They might believe you, but I know it was really about Justin Biber in a penguin suit...
m
0
l
a b 5 Linux
September 19, 2011 8:36:33 PM

audiovoodoo said:
...Justin Biber in a penguin suit...

Can't unthink that mental image, thanks alot.
m
0
l
a b 5 Linux
September 19, 2011 9:00:22 PM

audiovoodoo said:
A quick note on the RPi. It's boot loader is a little strange in that it actually uses the GPU for initial boot before resetting and passing control back to the main CPU. There was a little discussion about it on the forums but it RPi is your intended target then take a good look at the HW before you start shaping your solution to much.


[looks at documentation]yeesh, a true engineer's solution to a problem forced on you....

I guess the good part is that it could conceivably chainload onto a u-boot such that the only difference from TI's offerings is the location and content of the closed bits (ROM image on actual SoC -> firmware blob that is loaded by GPU)
m
0
l
a b 5 Linux
September 20, 2011 12:33:44 AM

I was aware of the... unconventional boot method. From what I've read however, you're still able to use a bootloader.

From what I understand (which I believe to be true), the GPU boots to a FAT partition on the SD card, which holds a module developed by the RasPi team. From there, the module points to the EXT2 partition on the SD card. You can have it load a boot loader at this point if you so wish.

At this point I'm just working on deciding on which packages to remove, keep, or add. Gonna start working on a theme as well, since that's easy enough to do without having the hardware.

Also, I got the armel version of Debian running in Qemu, and it only uses 9MB of RAM for the base installation! Just thought that was pretty amazing :D 
m
0
l
a b 5 Linux
September 20, 2011 5:54:09 PM

The RasPi dev team posted a video from Maker Faire, and they've confirmed the boot sequence. So at least I can worry about the OS itself and then just have the boot code point to a boot loader; neat.

Got LXDE installed and running in QEMU. It's a tad slow with 256MB of RAM, but still entirely usable. This is great news. I'm working on just slimming things down, and installing the applications that I want. I'm also starting to work on a theme.

Also, for amdfangirl found this for you if you're interested :p 
http://www.raspberrypi.org/2011/08/logo-competition/
m
0
l
a b 5 Linux
September 22, 2011 7:22:47 PM

audiovoodoo said:
There was an interesting interview with Eben the designer behind RPi on Slashdot just the other day:

http://interviews.slashdot.org/story/11/09/14/1554243/E...

I've been talking about it over on SemiAccurate in this thread:

http://semiaccurate.com/forums/showthread.php?t=5343

Whilst I can't stress enough that I do like the product, it still doesn't quite add up to me.


I already read that interview, but thanks for sharing it anyways!

What isn't adding up to you? Just out of curiosity. I don't think this project is perfect by any means, nor am I convinced that it will succeed how they're planning, but I just think it will be a neat toy. I've been looking for an ARM board for quite some time, just for something to fool around with, but they're all in excess of $100, often much more.
m
0
l
a b 5 Linux
September 23, 2011 1:04:26 AM

As I said on SA it's about the target market. When you read the /. interview Eben states that they expect the first batch to go to enthusiasts like yourself. If that is the case then why are they a charitable foundation as clearly the market is for enthusiasts? Next the HW itself. I've followed this from the start and the early device was flash drive sized and fitted with a camera, this has evolved to a credit card size device with rather limited connectivity. OK, so you get USB but where is the I/O without having to use USB bridges? Same with the video output.

Given that you need a whole load of peripherals and an SD card to do anything useful with it the price is really a little misleading. For yourself you have the KB Monitor etc but as an educational tool it just seems to miss the mark. As I said in my ramblings on SA there are quite a few Chinese tablets out there that have similar RAM/CPU specs but include a touch display and PSU for not a lot more. For education this seems to me to be a more flexible solution.

To my mind its a toy, a really cool little toy, but it just doesn't quite seem to know what it wants to be when it grows up. If you used it as the basis for a 2012 C64 like solution with a range of TV Monitor connections (I still think VGA would be the best option with composite for a bit of flexibility) and a couple of serial ports then it's a whole different ball game. How they expect a non boxed solution to survive students I don't know.
m
0
l
a b 5 Linux
September 23, 2011 1:09:33 AM

That's entirely understandable, and I agree with basically everything you say. I still just want one :D  For $35, I can just plug it into my monitor/TV to fool around with. Not a huge investment.

I have a tendency of obsessing about things for a short period of time, and getting distracted anyways. Who knows if I'll even still care by the time it's out :D 
m
0
l
a b 5 Linux
September 23, 2011 1:26:41 AM

A reasonably powerful device, low power draw, ethernet connection, at that price and I am sold (who needs a keyboard and monitor :) )
m
0
l
a b 5 Linux
September 23, 2011 10:11:09 AM

This is where I swallow my own argument and agree.

Not working at the moment my PC is on more hours a day than is healthy. I've disconnected unused drives, underclocked and dropped the voltages but it's still pulling ~110W of power. Most of my use is just surfing job sites and e-mail, all of which the Pi could do. But then I come back to where I started as I look at my PS2 KB and VGA only monitor (I don't own a TV) :( 
m
0
l
!