Thursday, October 28, 2010

View from the other side ... how do u like their company ??

Recently i came across this talk bye Joseph Nye on power equation , which led me into thinking how closely this maps to that most favourite grouse of ours(techies) viz. work culture at our places of work. An ideal place of work should be based on the 3 principles of a) Compassion b) Passion and their natural consequence c) Innovation.
a) Compassion - is at the root of it all. It is the love , respect and care that transforms a business relation(employer - employee) into something of greater value. Respect to the techies for their expertise, the value that they bring to the organization. It is obviously more than another department in the organization that just provides a utility at some cost. As Joseph Nye points out , there are 3 ways of having your will done i. By Coercion, threats etc. ii. By enticing the other person or iii. By making the other person want what you want . The first 2 constitute what is known as hard-power where as the last one make up that has come to be known as soft-power. This brings us to the next principle of Passion.
b) Passion - Programming is as much a mode of expression , as much a creative discipline as writing or painting is and as in other disciplines it requires passion and complete immersion of its practioners to produce the excellent work. Every shop that does "tech" must acknowledge this fact and should actively promote that culture and conversation in its DNA that addresses and nurtures this urge in its employees. In this context , these artists are the most valuable "asset" to the company and the trick is in aligning the companies goals and the creative fulfillment that the "artists" seek. The symbiosis is fulfilled in the acts of compassion for the employer that goes beyond the business relation and works for nurturing the talents of its employees beyond its immediate need and the employee that goes beyond his call of duty to bring value. It could be something anything from "20% project" , organizing a "friday tech talks" or any of such initiatives. No organization is small or big enuf for any of such initiatives.
Another manifestation of respect and compassion is by the way of involvement where the company seeks and promotes the views of its technical assets across the functions , creative as they are , the ideas that might be brought to table can only have positive spin-offs for the company.
Bottom line is unlike any other job this profession is not a business transaction at the end of the day where one renders some service for a price. There is a human side to this and a very valuable yet intangible transaction that needs to be settled as much as the tangible one , only difference being here the currency is of compassion and passion.

Sunday, October 10, 2010

new trends in data modelling...

This is actually a continuation of the earlier post. Although i haven't got the feedbacks and the answers i was seeking , the journey continues . I came across the following enlightening post from thoughtworks about the current scenary of data modelling , a great read indeed and a recommended reading to anybody interested in the topic.
Following are points i have gleaned from these and other topics :-
1. These are interesting times in history when the case for polyglot db architecture is slowly but surely breaking into the scene. The days of 1-size-fits-all solution of rdbms is over , now it is back to the future where each piece of data that an application interacts with has to be understood in context of its probable reads, writes , performance requirements, functional requirements etc. as, given the various sets of newer technologies that are maturing , there might be cases for various solutions other than rdbms .
As the article points out few of these cases are :-
i. static/lookup data (readOnly) - these are config data that are changed only at deploy times. Currently there are 2 solutions ti.e a) employed put them in .properties file and read them with java/cache them in jvm at load time or b) keep them in db if there is use case of having a configuration management software to update such data on the fly. So the data is mostly "read-only" or sometimes "mostly-read-only-rare-writes". So just to accomodate rare writes having to store them in a rdbms is an overkill.
ii. Semi-structured documents - This is a niche case , but doubt here is a universal one , how do you work with semi-structured data(undefined schema ) in a object-oriented language as java . I mean, to ask a very dumb question here , is how do u write the freaking models ?? and how do u write logic . Do u assume a schema with all possibilities ??
A related area is structured documents like xml that comes with additional facility to query. Its once again a judgement call of how much information can be left with the document itself and how much of it is to be extracted to separate db.
iii. Object store - Given the romance of ORM, where all u see are objects and by some magic they are persisted and whenever u see them u get 'em shiny objects . This is perfect if u never need to dissect the magic below by querying etc.
iv. Metadata or data that are to be heavily mined to glean information for reporting and other non-realtime requirements. The rigour of relational data here proves to be its undoing as the zillion of joins creates serious performance issues. Normalization were devised in an age where storage was expensive but given the situation today where storage comes rather cheap, its the response time that holds the key. Denormalized reporting tables is a traditional solution, and as the article describes graph db is suited for niche cases like social networking etc and i am sure there are other scenarios that can also be served by that approach.
Bottomline here is the store for such "non-realtime data " has to be segragated from the main transactional data so that these solutions can be implemented later.

This vision of future do have one caveat as i see it, the data of an application is distributed over many different systems and apart from very smart and versatile developers , you require a polished IT and deployment strategy to hold it all together.

And finally , to harness the power of these different technologies from a OO world , new paradigms are to be invented as also all these technologies have to come up with query language support/portability to survive and make it big

Thursday, October 7, 2010

ORM and beyond ..

My recent work involved writing something that is resembling more and more like a ORM implementation , like a deformed and abandoned sibling to Hibernate ;) . This in an environment where bitching about everything that's wrong with ORM makes for a good intellectual conversation and where NoSql is the topless bar that we as teenagers have sneaked into , tantalizing but not sure what to make of it .
So following is a list of issues that i have hit while working here , these are not exactly eureka stuff , these are common stuff that have been solved many times over . With the disclaimer that i am a novice in this i still document them as a note to myself as go forth and look into the wilderness for answers to some of them , will much appreciate if some of the readers can point me to the direction of solutions or more questions.

1. When i am writing a ORM based persistance layer it should be enuf , after the major models have been identified , to just have 2 methods for the models i.e save(Model mdl) and get(). is tat so ??

2. What is the granularity of the domain model ?? Given any domain it is always possible to have one super-grand-model that holds everything else like a company class that has departments that has ..... so any update on any employee on any department would still result in a save(Company cmp) . This is my understanding and my feeling is something is embarassingly wrong here , so wat is wrong ??
Now to spice things up a little bit more let us throw document-oriented dbs in how does all these things tie up. Documents to model ties up nicely with wonderful things like json , Jackson etc, but when things are large like u have a zillion departments and u have to update home address of 9999th user in 78976th dept u still have to deserialize the entire elephant (company in this example) and then iterate and change and persist .
Something is obviously very wrong here and i suspect the traditional object oriented view of data is the culprit , can anybody enlighten ??

3. When modelling a relationship what factors determine navigability , i mean as normal java guy it is most convienient to be able travel to and fro from any point in hierarchy . But should we enforce some discipline there ??

4. Getting back to the example scenario of point 2 above , say there is a composition we want to model like a Department having n people associated with it and say one among the n is a departmentHead a fact that we know from his role (role is also used by our security system). But the technical pain here is whenever i read a department object with thousands of people in it , and i want to know who the head is i have to iterate through the list of 1000 to get to it. There is an urge to keep a reference of this guy directly under the Department class that saves a lot of iteration but that is duplication of information also , but still this solution can be used if this is something we do at the programming layer and change db design for it . Any thoughts.

5. In enterprise apps space where the demands scalability is limited does the noSql,commodity sever camp bring any value or they are still going to stay old school for some time.

To be continued..