Uber Leet Hacker Force Radio is syndicated on Hacker Public Radio. The episode can be directly downloaded from here.

You can check out the code on Launchpad: http://launchpad.net/dornado. Next I will add proper Epoll and the HTTP part. But in the meantime, you can try it out with this demo code snippet:

private import tango.net.device.Socket;
private import tango.io.model.IConduit;
private import tango.net.InternetAddress;
private import tango.io.Stdout;
public import ioloop;
 
class HelloServer {
    private ServerSocket sock;
    private char[1024] buffer;
 
    public void handle_connection(Socket connection, char[] address) {
        connection.read(buffer);
 
        connection.write("hello world!");
        connection.shutdown();
        connection.detach();
    }
 
    public void connection_ready(ServerSocket sock, ISelectable.Handle fd, uint events) {
        while(true) {
            Socket connection = sock.accept();
            connection.socket.blocking(false);
            handle_connection(connection, "");
        }
    }
 
    public void call_connection_ready(ISelectable.Handle fd, uint events) {
        connection_ready(sock, fd, events);
    }
 
    public void start() {
        int port = 3000;
        int max_waiting_clients = 128;
        bool reuse_address = true;
 
        this.sock = new ServerSocket(new InternetAddress("0.0.0.0", port), max_waiting_clients, reuse_address);
        this.sock.socket.blocking(false);
 
        auto io_loop = IOLoop.instance();
        auto callback = &this.call_connection_ready;
        io_loop.add_handler(this.sock.fileHandle, callback, io_loop.READ);
        Stdout("http://localhost:3000").newline.flush;
        io_loop.start(sock);
    }
}
 
public void main() {
    IOLoop.use_epoll = false;
    auto server = new HelloServer();
    server.start();
}

You can compile this with:

ldc helloworld.d ../ioloop.d ../language_helper.d


Over the past few months, I’ve really gotten into packaging on Fedora and Ubuntu. I’m not an expert, but I can genuinely say our packaging situation is crap. We are so close. And yet we ended up with two major implementations, that are almost the same. There is no significant difference between RPM and Dpgk.

So will Fedora abandon their 15,000 packages and choose Dpkg? Will Debian abandon their 35,000 packages and choose RPM? No. Neither will. There is no way for them to transition. And they will not give up their self branded solution. The only way forward is to make a new system, that is both better, and backwards compatible with RPM and Dpkg.

That is what I have been toying with for the past few months. It is called Packagetastic and It is a replacement for Dpkg and RPM, but still backwards compatible with them. Right now it is just a proof of concept, but can build many small packages on Ubuntu and Fedora.

I made a screencast to show how it works. This is just a simple, introduction. More example on how to use it will come soon. You can view the screencast below:

packagetastic_1.ogv (96.2 MB)

More information about the project can be found here: http://launchpad.net/packagetastic

Please provide your criticism and feedback at the mailing list, this blog post, or my identica page.

If you are interested: Rester is a fast RESTful web server and framework written in D. It should be as easy to use as Ruby on Rails, but provide good performance by default, and be easy to deploy.

I made a brief screencast, with an introduction and an example that uses server push.

rester_1.ogv (27.7 Megabytes):

You can also view the video in the legacy flash format at Youtube:
http://www.youtube.com/watch?v=uBfdn5RCrlw

Feel free to try it out yourself, and leave comments.

bzr branch http://intuitive-desktop.org/bzr/main/

Merging is SOOOOO much easier in Bazaar.

Also. I stumbled over a tool called Meld Diff that is very useful when you need to perform complex merges that can’t be performed automatically.

Look at the pic. You can click the little arrows to merge each section, one-by-one.

I’ve been researching problems in the current desktop paradigm for a few years now. I even carry a list of said problems around in my pocket. Throughout the years there have been a few problems that have driven me mad. I have seen them reflected in so many dying projects. I literally have dreams about them, where I’m at a podium in front of millions of developers, trying to beat them in the head with the most critical problems.

By far, the most critical problem is: “The barrier to entry for contributing to other projects is too large. If you want to contribute to someone else’s project, you have to spend hours or days reading their code, and trying to find out where everything is and how the ‘program flow’ works.”

For example, recently I had a few hours and decided to spend them looking at the code for Network Manager, to see how I could add support for EVDO laptop cards (I was using a Verizon pc5740). I decided on two goals for that sitting: first, to figure out how NM uses HAL to get information on networking hardware. And second, to figure out if HAL worked with EVDO cards. I downloaded the NM source code and started looking in the src directory. I had the immediate observations:

1. Where is the start of the program?
2. It looks like this is actually a daemon and an applet, possibly more
3. Most everything, header files, source files, and make files are dumped into one directory
4. I remember why I haven’t used C or C++ in a few years, on my own projects
5. Has anyone on this project even heard of Model View Controller?
6. Why the hell do Gnome developers still use C?

Those are some dirty thoughts. I continued to dig. I found main and started reading. Still not knowing if what I was looking at was some sort of service or the Network Manager Applet. Next I noticed the naming convention used. It seems that they are using the naming convention of giving everything a prefix such as “nm_” to simulate name spaces. There is no context of where the function or variable comes from. Because the directory and file structure does not reflect the namespace, I had to search through many headers to find out where functions and variables were coming from.

Once I finished main, I decided to just open a bunch of the other source files, and look at their code. The problem I encountered was that there was no context to where this piece of code would fit into the program flow. In order to determine that, I would have to go back to main, and trace the entire flow of the program. I had no desire to do that, so I moved on.

Next I decided to just look at the Models, because they would have to be networking hardware. I could then find out how the other hardware was accessed and then see if an EVDO card could be hooked up the same way. But of course that didn’t turn out. There was zero separation of data, logic, and user interface. If they had used Model, View, Controller, this would have been much easier.

So a few hours had passed, and I was about half way through my four hour block. I decided to look at one of the small examples to see how this program could be easily used in a few lines of code. There were no examples.

I used beagle to look for the dbus and hal calls to try and find the “magic” of how the program worked. But as usual, almost every function used dbus and hal. It was nearly impossible for me to figure our the “flow” of how either dbus or hal were used.

Ding. Time was up. It had been four hours. I had made zero progress. I was pissed. Pissed at how Gnome worked under the covers. And pissed at myself for not understanding the inner workings of someone else’s mind. I closed all the gedit tabs and deleted the Network Manager source code.

Now, what were the problems I encountered?:

1. It was very hard to track the program flow
2. There was little context to any of the code,
3. Code namespaces and class names were not reflected in the file and directory names
4. There were no example programs to help me get started
5. It was difficult to find main
6. There was no separation of Model, View, and Controller
7. There was little separation of layers of abstraction with dbus and hal code everywhere
8. The limitations of the C language were hampering my progress. IE: No unit tests, no namespaces, cryptic make files, and evil header files

Now we could have solved those problems by changing a few things:

1. Make it easy to track program flow, by using Model View Controller, and having the file hierarchy reflect the code namespaces
2. Use a programming language that supports unit tests, object orientation, and importing code rather than header files
3. Have a few example programs that show how to do very simple stuff. The more trivial, the better
4. Sticking to Model View Controller would have also solved the abstraction problems, as the dbus and hal stuff would naturally gravitate into Models
5. Having a unit test friendly language would have made it easy for newcomers to know if they screwed anything up

Will any of these changes be addressed in Gnome? I don’t think so. Things may get a little better with Vala becoming popular. Despite that, any Gnome developer reading this will probably think I am an idiot, that just needs to learn the ropes. Which is exactly the problem. The way things work is the barrier to entry. I keep harping on Gnome, but KDE has the same problems (It is astonishing how similar KObject and QT is to GObject and Gtk once you spend some time with them).

To answer my question, I can’t contribute to many Gnome projects in 2 to 4 hours because of the way things are developed, and the way the code is organized.

Usually when people write their unit test, they are just trying to get them to work. The point of view they have is the Test Case Point of View. A developer with this point of view will have these goals:

1. The framework needs to be robust enough to satisfy all the unit tests
2. Once all the test cases are covered, and pass, the code is done

Now most developers would stop here, without moving on to the next stage. But they would discover that their framework was hard to use, redundant, or unnecessarily wordy. Or worse, only their users would find that out.

The next stage is the Usability Point of View. In This stage, the developer should make many example applications using their framework. This includes “Hello Word” applications as well as large apps that try to test the limits of the framework. And Don’t forget cloning neat little apps, like Cairo Clock or Tomboy. While doing this, they should try and focus on these items:

1. Too much configuration or the innards of the framework being exposed for no reason
2. If you end up writing unit tests that test both the application and the framework, this is a sign that there is too much overlap in parts of the framework. Remember Model View Controller.
3. If the framework is very wordy or has a lot of options that should be consolidated, that can cause example programs to bloat up and get very long.
4. There should be very little coupling. If every time you add new code, you have to add multiple dependent pieces of code to deal with that one new thing, that is called Coupling to The Nth Power. That leads to a framework that grows exponentially, and is concrete, near impossible to change, and usually requires a lot of code to do even simple program. Examples of this are The .NET Framework, and J2EE/JBoss.
5. If you run into a problem that requires a change in the framework, and it would be “very unpleasant” to fix that problem, that is a bad sign. Everything should be as easy to change as possible.
6. If you are having to write a ton of comments to explain some voodoo that is happening in the framework, that is a sign that the naming in the framework is confusing, or that there is too much overlap in the classes.
7. If you are not enjoying yourself while making the framework, that is a bad sign. That should be fixed too.

Now that is a pretty large list of things to look out for. I would say the one that I often run into is number 3. I always end up having too many options, or overlapping options in my classes. I even ran into that problem this evening, (which inspired me to write this) while working on a Cairo Clock clone.

Just for fun, I will go over the thing that caused the example to go wrong: I was writing the part that uses data binding to attach the SVG images to layers in the Drawing control. Now the Cairo Clock has a lot of layers. I think it is 12 layers. So for each layer I had to write a binding just to connect it to the SVG file. The binding looks like this:


<Layer name="clock_drop_shadow">
<Binding name="image"
model="ClockImages"
view_properties="image_file"
model_properties="clock_drop_shadow_image"
on_model_change="save_changes_to_view"
on_view_change="do_nothing" />
</Layer>

Basically we have a Layer with a Binding. The Bindings job is to connect the View to the Model. This one connects the ‘image_file’ property of the View to the ‘clock_drop_shadow_image’ property of the Model. It connects them so that it automatically updates the View when the property in the Model changes. Now this is a great way to do data binding, much better than anything that is out there. But as said above, it is too much code for something that will have to be used many times in each object. Not surprisingly, this revelation has come up by writing an example that requires a lot of data binding.

Now to fix it we can do this:


<Layer name="clock_drop_shadow">
<Binding name="image"
bind_direction="view_to_model"
property_map="image_file:clock_drop_shadow_image" />
</Layer>


This looks good. It solves the duplication problem of having two properties that are used to say how the binding direction flows, and the problem of having two properties to hold the way we map one set of properties to another. It also uses fewer lines, and could even be squished into one line.

So it worked. We went one step further, and used our Usability Point of View, and now we were able to make the code more usable, where we would not have seen a problem. So keep those two points of view in mind next time you are developing a framework. And if you have any tips on improving this concept, or have any success stories to tell, email (mattjones@workhorsy.org) me, because I love this shit.

I started with C#. That went down in flames because of the huge overhead in startup time for large assemblies. It also did not have all the code implemented on the Mono side of things. I wanted an easy-to-use language, with all the great Linux technology, like DBus, Cairo, Hal, and GStreamer.

At that point I decided that I was being too much of a wussy, and was going to use C/C++. That quickly crashed and burned as I re-learned why I hated C/C++ in the first place: “Standard Libraries” that are not standard or consistent across platforms; No standard libraries for Lists, Array Lists, and Hash tables (yes. I’ve used STL. It is very fucked up on Ubuntu and Fedora Core); Threads were not “Standard” either; Endless fighting with header files to make the compiler happy; And finally the nightmare that is USING Gobject. It all amounted to too much work, and almost no progress.

My next attempt was to use D. Which was, and is very promising. It is as fast as C, and includes all the threading and collection support. The problem however was that the Gnu D Compiler was not packaged in Debian, and didn’t include the standard ‘Phobos’ library. That was fine. I fought with it and got it working myself. I was cruising along and creating the Postgresql database back-end when I hit a wall. The wall was that the promise of ABI compatibility with C was not entirely true. It was true for the Cairo wrappers I made, but not for the Postgresql wrappers. So I decided to wait a bit longer and see if a solution presented itself.

A solution came a few months later, after I spent a while learning Ruby on Rails, and gained the wisdom that comes with not focusing on performance and over configuration. I decided to try and implementing everything in Ruby. Simply put: Ruby is amazing. Once I understood the Model View Controller technique. And I mean really understood how it works. Not like the outright lie that is in Wikipedia article. I had an epiphany: If you break your programs up into user interface, logic, and data, it makes everything so much more streamlined. All the problems with coupling and optimization simply vanish. Well it was (and still is) sufficient to say that I would do the entire project in Ruby.

But then I started to notice things. Like the amount of memory even a simple Intuitive Document was using. This wasn’t really a problem. But it would be for small machines like the n800, and would inevitably be a problem when the entire desktop consisted of 6 programs that each use 20 to 30 megabytes of ram. We would have all kinds of issues with paging memory to disk, and the CPU waking up like crazy to manage it all.

I decided to wait a bit and see what the community came up with. This was a long stretch where things like Lowfat, Pyro Desktop, and Online Desktop where being revealed. So it gave me hope that someone else would want to write libraries in a high level language.

Then very recently, I started seeing blog posts about this new GObject friendly, C#-like language called Vala. Like this post where a person makes Gtk MozEmbed wrappers for Vala in a few lines of code. This is how GObject should be used! Since I am using a lot of the Gnome infrastructure under the covers, this might be a very good choice for a library-level language.

So this is the plan so far: Once we get more stable, someone will rewrite the Intuitive Framework to use Vala. Then we will need a way to still use Ruby for the Intuitive Desktop and application part.

Sounds good. Now we just need more contributors!