Category Archives: programming

Backbone everywhere

internet, programming, ,

I finally put my newly built Node.js MVC stack on github! You can download it here: backbone-everywhere.

What’s so special about it? Here’s my list of exciting features:

  • Pages are rendered on the Node.js server by Backbone and jQuery.
  • All script resources are bundled by browserify and fileify,  uglified by uglify, and gzipped by connect-gzip for fast loading and deployment on other possible javascript environments.
  • The entire Backbone MVC stack works on the server, and is loaded in javascript enabled browsers to take over from there.
  • The app state is reflected in the url by means of HTML5’s pushState, or using hash notation when not supported.
  • The same app state is regained for non-javascript browsers pulling full requests from the server, so no worries about SEO!
  • All client / server communication is handled by socket.io (ajax is sooo 2009) and subscribed clients are updated with published models.
  • A JSON-RPC service listening to ‘/api’ requests, with an easy to build on service layer. Handy for non-web contexts such as mobile devices.
  • All data is persisted in Redis through an adaptation of mine of backbone-redis, enabling indexing, sorting and  foreign key lookups.

For me this is a whole new approach at engineering web applications, but I think I’ve managed to get a grip on it.
Not only that, it gave me a great impulse to reconnect with the pioneers of tomorrow. Because what I have done was build on top of the stuff from people with great vision.
Big shout out to the open source community, and the people willing and wanting to share. The sum of it’s parts will eventually overcome the current patent trolling paradigm.

What are you waiting for? Dig in!

Node.js is the future of web dev happening now

internet, programming, , ,

Having been a javascript fan since I started working with it a long time ago, I immediately fell in love with Node.js. Having discarded Jaxer earlier as too proprietary, even though it offered a solid mechanism for code reuse on the client, I have now adopted the uber active Node community.

You see, me and my partner decided to build a new and exciting community site/app (of which I have to keep the details secret for now). So when I started designing our new web 3.0 app with accompanying mobile app, I thought about and thoroughly  investigated the possible frameworks out there. We decide to build a single page application that would work in all javascript clients, and would gracefully degrade to server roundtrips when javascript was not available (but also to enable deep linking for SEO). I quickly decided to discard most PHP frameworks for our MVC setup. Not only because of the fact that those were mostly too bloated or complex, but also because it implied having to recode a lot of functionality for the client. Of course I also favor the stateful and event driven possibilities of javascript, so I made the paradigm shift and chose the Node javascript stack.

With all the Node modules out there it is finally possible to create a full MVC framework operating on the server as well as in the client, in the form of backbone.js with the help of bones (a Node backbone server implementation offering code reuse in the client).
And with the help of HTML5’s new pushState, we don’t have to worry about breaking Google’s need for deep linking. (Those looking for an example with gracefully degrading URL’s, take a look at the jquery address plugin.)
More goodies come in the form of the browserify module, enabling us to optimally pack all our resources for client side usage, even all our templates and other static files!

Some more info for those interested:

With regards to storage, we decided to go for Cassandra, since we expect a lot of writes, and a lot of scaling. In the mean time I am hoping somebody will come up with a nice abstraction on top of the new Cassandra CQL language, since there are already some Node modules out there working with it.

Taking sessions into account, I am currently favouring Redis, which also has a nice pubsub layer. But I haven’t investigated that path fully yet.

In the mean time I am working on an iphone demo in Appcelerator’s Titanium. Too bad it doesn’t support all the functionality we need on Android as well.

That’s all for now. I have to try and curb my enthusiasm, as all this goodness may come at the expense of my sanity due to sleep deprivation.

My love for Paypal just died a little.

internet, programming

After having put some days of work into creating my friends book site (iktami.com), I decided to keep it simple and implement Paypal’s “Add to cart” buttons.
Wrong choice.

You see, I was under the impression that Paypal’s shopping cart would be able to handle shipping cost based on weight. But no, it isn’t. Well, only if you’re a merchant residing in the US.

What? I thought computers and the internet would help bring an end to discrimination and suffering.

What could be the reason for this? I am sure it’s not a technical problem, because that little and simple piece of code is agnostic of location. And it isn’t money, because they would probably make even more if they didn’t piss off merchants outside the US.
Maybe they just don’t care anymore, having all that money, and getting (re)tired of it all.
Or, could it be some political hidden agenda? These things pop up everywhere these days.
I am probably missing something here.

Taking PHP development to the next level

internet, programming, , , ,

A lot of my enthusiasm for programming comes from my never ending interest in it’s evolution. I played with computers since I was 7, and already knew at age 13 that I wanted to become active one day in the ‘automating’ industry. I’ve always regarded programming as a trade or tool to automate repetitive tasks, but also to offload (error prone and inconsistent) people tasks to fool proof automated systems. I just wanted to make stuff easier and better, and found a way to do it.
I truly started my programming career in 1999 as a Java programmer in the corporate world, learning a lot about real world software problems. It made me realize a lot of things.
But even in the year 2000 internet applications were not widely recognized as “the way to go”. My employer, one of the best amongst Dutch knowledge engineering services, did not pay full attention to it yet.
So I left. I simply had to find out more about web technology, which I regarded as the future of software development.
I carefully chose to follow the ‘open source’ path (as opposed to the closed MS world), and divided the territory into “corporate vs fun and dynamic”. I ended up in PHP land.

But soon I found out about the drawbacks of PHP development. It’s a loose typed language, with a scattered user base, supporting all kinds of immature frameworks and libraries, for all the wrong reasons. Simply stated: PHP land was short on inhabitants with extensive knowledge of software development in general. After a while though, PHP was recognized for its RAD abilities, and more and more people with skills came into the game. This lead to a myriad in yet more frameworks and libraries, but now it was all about documentation, participation, and a growing user base. Because solutions were being reviewed, tested and explained, quality started to drift to the surface.

But how does one recognize quality in this field?
Since I have never studied software development on a university or such, I will always regard myself as a noob on matters that I have no experience with. I have learned everything by reading books, and from material on the internet. But because I was aways so insecure about my knowledge, and truly interested in what I was doing, I never stopped reading. That’s how I discovered that it’s most beneficial to start learning the industries’ best practices and ‘patterns’, before formulating your own suspicions and concepts regarding programming solutions. Building onwards from that premise I feel I have finally grasped an overall intuition for what matters most, and which choices should prevail others. Of course there is no holy grail of software design, but given a business goal, context and limiting factors, one surely can sketch a fruitful path for future development.
Do I bore you already?
The point is that I think that higher level principles should be considered first, when making choices in software development. Therefore I was always drawn to development platforms that obey such principles, even though some of their subsolutions are plain awful.
Steep learning curves and incomplete documentation were arguments very low on my choice break down list. As I have a brain, and a debugger, I’d rather work with beautiful flexible software than some simplifying framework caving me in and making me jump hoops to do the in-ordinary.
(Please don’t respond with PHP framework ‘showdown’ arguments, as I think Sheldmandu has a nice take on that, even though I don’t follow his simplistic view on ‘inheritance is key’.)

That’s why, about 4 years ago, I chose to go with the Zend Framework, being the most mature and offering so much of what I thought was important. It still forced me to do a lot of work by hand, and a lot of their code base seemed rather inconsistent. I think mainly because they let it grow too quickly without proper reviewing user contributed code base additions. And as a result of that, I think a lot of their solutions are simply not well thought out (like forms and data access).
But I loved their coding style, and the way they used naming conventions to get stuff rolling.

Then, about two years ago, I stumbled upon FLOW3, a new PHP framework built by the developers of TYPO3. In my opinion it will be the most advanced and productive PHP framework in the near future, but that has to be seen as it needs to become widely adopted. It has implemented a lot of the best practices from the Java scene (annotations, AOP, dependency injection). Which I think is a good thing, as that scene has been dealing with the largest chunk in complex software development for a long while.
I concluded that it needed a lot more time to come of age.

Around that same time I started using Magento for my own webstore (project was killed over a refuted business deal), and after that I worked freelance for 2 years with it’s complicated, yet powerful CMS abilities. A lot of 3rd party extensions built by novice Magento developers were indirectly compromising a lot of the projects I worked on (in the form of performance degrading or bug introducing plugins), thanks to Magento being too flexible and not understood well enough.
But Magento made me realise something:
What’s the use of a framework or library, if it doesn’t offer concrete reusable solutions to real world problems? In other words, why could I not find a development platform crafted on top of these high quality frameworks and libraries I so much admired?
Magento at least offered me out of the box configurable layouts, views, themes, multi site functionality….bla bla, etc etc.

Now that I wish to work on more versatile projects, preferably using out of the box, proven solutions, I have to check my facts again. I don’t want to go back and reinvent so many wheels. What are my options? What CMS can I use to not have to repeat myself too much for every similar project?
Let me write down some of my requirements:

  • An intelligent caching layer, like Magento’s, with it’s hole-punching capabilities.
  • An annotation-based domain persistence layer, using POPO’s (plain old php objects), decoupling data storage and making persistence trivial. (Doctrine 2 might fit these needs.)
  • An easy to setup service layer or domain API, enabling easier development of client-server architectures. Javascript, apps and flash are my main clients these days.
  • Straightforward creation of sites, templates and content, facilitating a multi disciplinary approach and separation of concerns.
  • Inheritance based themeing, allowing a stack of theme overrides, like Magento’s upcoming themeing system.
  • Plugin based architecture, preferably with a controlling mechanism warning developers about slow and impeding functionality.
  • Many great working plugins fitting my business needs!
  • Possibly some elaborate publishing mechanism, allowing for the entire application (files AND data please) to be pushed onwards in it’s development cycle: DEV / TEST / USER ACCEPTANCE / PRODUCTION
  • A large user base and developer involvement ensuring stable releases.
  • …

(Please feel free to add to this list)

Lately I found that FLOW3 is approaching it’s first stable release (possibly september 2011). But not only that, the release of TYPO3 v5 will be based upon FLOW3. So I quite hope TYPO3 to be the CMS I am looking for.

Group Messaging needs to grow up.

apps, internet, programming, ,

I recently developed the need for a real time group messaging system that allows the members to subscribe to topics and tags. Posting messages in such topics or including such tags would then result in a push notification to the subscribers mobile devices.
The admin of the group could create a taxonomy for the groups context, and members could contribute to the taxonomy, based on an approval system.
This allows for subscribers to carefully decide themselves what messages they want to receive, instead of somebody else deciding for them that they are interested in.

You see, because I hang out with a loosely organised group of artists, freelancers and randomly creative individuals, we need a tool to be able to mobilize each other, but only the people interested in the joint venture at that moment. So we want to create a list of topics/tags and locations to subscribe to. We want to keep informed of only what we subscribed to. The group is just too large to blindly spam everybody with every message.
And since we are often in the pre-organising fase, we simply want to be able to tweet a message including one of these tags to elicit interest and motivation to those possibly wanting to cooperate.
Not only that, sometimes we simply want to mobilize a group of individuals to meet at a certain location. Last minute spontaneous action.

Well, after doing some research I found some great and promising startups in the group messaging industry, as that industry is taking off like storm.
I really like the irc-like Convore, and posted my feature request there, but no answer yet. Too bad none of the tools I looked at (Beluga, GroupMe, Kik) offer what we need. Sure, there are some forum-like tools that send you an email when somebody posts in a thread, but none of these tools allows for subscribing to a taxonomy. Please correct me if I’m wrong.
I think most of these tools are truly helpful after the fact of organising structures. When all the roles are defined, the players are known, and from there it’s just a matter of communication and collaboration.
Not what we need.

So, because I am a programmer, and therefor lazy by nature, I want to appeal to the group messaging industry first. By asking them to incorporate such a mechanism in their platform, so I don’t have to build it myself.

Anybody?

Magento’s dataHasChangedFor(isDirty)

programming,

I just built 4 hours of code on top of isDirty, when I noticed it isn’t even working! I wanted to test if something has changed in the ‘customer_save_before’ event, so I could do something first. But Magento seems to simply not care if an object’s main properties or relations are dirty. It just saves whole object trees, wether something changed or not. AAAAAARGH! Yet another performance penalty, straight from the hip.

Isn’t it basic stuff to have your value container flag properties and itself as dirty? Magento started on isDirty and flagDirty, but didn’t pull through and left it unfinished. Instead they implemented a dummed down method called “hasDataChangedFor”, which doesn’t allow for testing if the object itself is dirty or not (because one of its properties is). I notice that it’s sparsely used, but not for testing whole objects before they’re saved.

Now I have to create an unnecessary bloated mechanism that goes through each property in getData, and see if it is found in oldData, and then test if it’s changed with hasDataChangedFor. AAW!

I hope the Varien guys pick up on this and start using the dirty pattern, because it’s simply too much to carelessly save whole object trees and make costly database queries when it’s not even needed.

Maybe I am too much of an angry guy right now, instead of a quiet and thorough investigator, so I might have overlooked something here. I hope so. Please tell me if I’m wrong 🙂

UPDATE: it’s even worse than I thought, because in both customer_address_save_before/after, _origData is not populated, so I have no clue wether something about the object has changed or not. Bummer! Does that mean that I have to load up the old objects from the database first? So much overhead, I am getting more even depressed.