The Cathedral and the Bizarre

Good post here. It's thesis: that in the near future the software industry will look a lot like the energy business does today with its regulated and unregulated business; where regulated business is open source and its predictable steady stream of income and where unregulated business is analog to highly vertical, niche software products that are closed, closely guarded and expensive.

Interestingly, the software industry as we know it grew up with unregulated businesses - and only in the last decade has open source created these credible alternatives. I think the future looks similar to the energy industry: large technology companies will have a mix of regulated and unregulated businesses, that maximizes the advantages of both. For standard, widely-used technologies, open source "regulation" makes sense because it lowers development costs and provides a standards-based, predictable subscription base of business. For niche and high-end software, companies will still expect a substantial return on their development cost, and therefore will protect that IP and sell it at a premium until competition makes that impossible. The most successful of these integrated companies will be careful not to exploit the community, and will be respected for having transparency between what parts of their business are "regulated", and which parts aren't.

I certainly see this as the future of Microsoft which is unlikely to go completely open any time soon.

A Public API and Policy Would Really Help

If you have ever worked VMS or Windows at the API level you are familiar with the concept of a "Public API" and the concept of an API lifecycle. A public API is one that consumer code can count on. Some code is "for internal use only" while other code is meant specifically for "public consumption. " The public API code is a contract with the outside world. An API lifecycle is a contract between the API and its consumers that provides a level of backward compatibility and stability to the API. API calls never simply change or disappear. They are deprecated and then supported for some period of time before actually being removed. DEC had a really strong policy. DEC API calls would first be deprecated, after a number of releases they would be removed from the documentation but would continue to be supported, and finally after yet another number of releases they would be removed from the source base. Customers were given plenty of warning and time to port their consumer code.

Example:
Windows NT used to have a DLL called NT.dll. All of the actual system calls existed in this library but Microsoft made no commitment to anyone who called its methods directly that they would work from one patch or upgrade to the next. Developers were told specifically to avoid NT.dll and instead to rely on WIN32. NT.dll calls were at liberty to change as often as Microsoft saw fit, while the WIN32 library signatures had to maintain consistency for customers. Microsoft would then take the responsibility of mapping the stable WIN32 API calls to the more volatile NT.dll signatures whenever a call needed to be changed.

Many open source projects do not follow this practice. Some do, I believe Apache Struts for example deprecates for major two releases and removes on the third. As far as I know Alfresco does not have such a policy but I really wish they did. Enterprise customers expect reliability. They demand that versions come out at a slower more digestible pace and I would bet they expect some stability and policy around the backward compatibility of the API.

Today I installed a AMP in 2.9B (community) that is fully functional in 2.1E (enterprise) but does not work properly in 2.9B. The source of the problem is due to a class that has been moved from one package to another. This sort of refactoring happens all the time in source code. It's really important and it has to happen -- but we need to maintain some backward compatibility at the same time.

It may be a semi-valid criticism of this post that I am trying to install something that worked in the enterprise version but breaks in the community version because the community version is a sandbox / lab environment. I would however disagree. The labs release cannot be entirely free to evolve without consideration for any backward compatibility if we expect to build a community willing to contribute plug-ins and add-ons, etc. At OSBC many of the executives, when asked why their developers were taking from open source more than giving responded that it's simply too difficult to package and maintain contributions. This issue is further complicated if you cannot count on compatibility for some reasonable time frame.

There are downsides to maintaining backward compatibility. Backward compatibility code is cruft and can, over time, become weight on the system that impedes innovation. This issue can to some degree be managed by proper factoring of code and packaging as well as policies that attempt to seek balance. Don't try to remain backwardly compatible for more than a reasonable duration. Give customers the ability to port their code forward over the course of one or two upgrades but no more.

Stability and predictability goes beyond releases. If we want to encourage development outside the walls of Alfresco we have to extend some stability to the source code as well. Developers want to know exactly what API calls are public. Additionally they want fair warning when public calls change as well as to be given a sufficient amount of time to port their work.

Repository

During OSBC I was able to catch up with a number of fellow Alfresco customers. All of us are eager to gain a better understanding of how Alfresco will proceed with its repository harmonization. During the Alfresco Community in New York repository harmonization was a major theme during the Ask the Experts session. Clearly it's an important area of concern for customers.

Alfresco's road map on the wiki states the following priorities for 2008

• New and enhanced team and enterprise collaboration services

• Publishing and management services for dynamic, Web 2.0-enabled websites

• Web client usability

• Scalability, performance, and enterprise-readiness

• Standards

It's difficult to tell if repository harmonization is included in these initiatives or not. Clearly it is not spelled out as a explicit initiative unto its own. I am looking forward to the MuSCoW roadmap / release schedule we were told about at during the User Meetup in San Jose. MuSCow stands for Must do, Should do, Could do, Won't Do.

I had a thought today that I need to follow up on. We have been talking about "Repository Harmonization." I, for some reason assumed this meant that the two repositories will be consolidated in to a single solution, taking the best features and capabilities from each. This is a probably a question of semantics -- but I got to thinking today: "harmonization" is not by definition consolidation. Harmonization implies two or more entities acting in harmony (for example two guitar string vibrating at agreeable [very technical -- I know] frequencies.) I am still under the impression that harmonization means consolidation but I want to follow up and make sure. Clearly the combined feature set is important to the customers that I've had the fortune to speak with.

Acquisition Motivation and Customer Interests

At OSBC this year a panel discussion was added to the agenda called “Can Open Source Communities Survive Mergers / Acquisitions?” The panel was moderated by Robin Vasan, and included representatives from MySQL, Zend, SleepyCat, and Jboss. This was one of the best talks I was able to attend.

Zack Urlocker of MySQL (now Sun Microsystems) said something that resonated with me. He said when they [MySQL] were acquired by Sun it was because they saw Sun as an environment where they could grow MySQL at an accelerated pace. MySQL wasn’t simply working towards an “exit." They were/are building a big, customer focused software company and this exit was the next logical step.

I really appreciated this concept. I don’t care for the idea of companies who like the beef industry, which, raises cattle for the harvest (slaughter); build businesses just to be sold. The motivation in my opinion is wrong. I understand the pure business justification. This course of action makes perfect sense for the entrepreneur but I question the value and effect it has on the customer.

Customers want the focus on their interests. When it comes to companies investing in software; they want to know that the company is A: focused on the product and it’s ability to meet customer needs, and B: the company is going to be around for a good long while. An open source company building itself simply to be bought; that is, a company which has the primary objective of reaching an exit is in my opinion at serious risk of accomplishing neither because the motivation directs the focus elsewhere.

Acquisition may be the right move but I think the motive behind getting to that point is of great importance. In general I think exits aught to be thought of as new beginnings on sronger footing. When exits are simply exits, it is a red flag for the customer. Customers, Customers, Customers – this is what matters.

Open Source and "Commerical Interests"

Lorenzo E. Danielsson has an interesting blog post here, which responds to this post. I wrote the post on the 26th and later edited out some of the things I thought were off topic which was:

Open Source companies aught to be able to grow their business to the scale of organizations like Microsoft. On the way there we stand to learn some things from companies like Microsoft where they have (very limited at the moment) experience with going open. Is it good or bad for society when companies get this large? IMO, it depends -- transparency matters at scale.

In fairness to Lorenzo and Alfresco I need to make clear a few things and also make available the text, which Lorenzo has commented on. I stand behind what I’ve said; I just didn’t think it helped the post in terms of staying on topic.

First. I don’t work for Alfresco. I’m a community member. My opinions are my own.

I absolutely believe that open source has the potential to create both economic and vast social benefit. Apache, Eclipse, Linux for example have created a feedback in the economic/social ecosystem that without open source simply would not exist. The kind of productivity that free and open source software produces is very special and very important. It has the power to change the world and it is doing so.

I like many have trepidation with large amounts of centralized power but this is because “absolute power corrupts absolutely. “ We can't count of the benevolence of individuals or organizations because it is almost always temporal. In our industry IBM, Microsoft, Google have all struggled or continue to struggle to maintain the balance of scale and public perception. Be transparent, focus on the customer.

To address some of Lorenzo’s points:

Lorenzo commented on this text I edited out the other evening:

Open Source is not about the bits, it’s not about the community, and it’s not about licenses. It’s about a better way to do business (read: make money via serving customers.) Have you read the Clue Train Manifesto? Does it sound familiar? They don’t have any bits, community or licenses. They only have a model based on the fact that A: the world is massively connected, B: the fact that customers are as informed and in some cases better informed than vendors and C: the fact that the market now has the capability and the expectation to engage in bi-directional or n-directional conversation with vendors and other customers. They have a model that sounds a whole lot like open source without all the distracting implementation details.

Lorenzo responds:
It’s not about community, because community implies those ungrateful masses who are supposed to serve the elites. The elites have built up a system called wage slavery which is how things are supposed to be done. Community is the start of dangerous dissent against the ruling order. Never mind that those communities wrote the software. Once the labor is done, business can take over.


The System that is Open Source would not survive on community alone. “It’s not about community” is meant to say that open source is not a hierarchy of components but a network and that community alone does not make open source what it is. Community is a very important aspect of Open Source – however it IS NOT what open source IS. Community is a significant node in the network in that it can push both negative (regulating) and positive (re-enforcing) feedback on to the network. I like Lorenzo believe that the community should be strong via its rights (example: right to fork, right to vote on direction) and diverse.

Lorenzo further commented on this text I edited out the other evening:

The software industry has historically been abusive to its customers. Open Source promises to focus on the customer and that inclusive conversation. So source code, it turns out, is in a big way merely the olive branch: a statement of commitment to the claims we make. Don’t believe me? You probably use MySql or Apache. Do you care about the source code? Almost none of us do. I would be sorely upset if I found my development staff was hacking MySql code. I want my developers in the community, I want them to contribute but source code has very little to do with anything on the business side. I do care that MySql is open source. It has nothing to do with access to the source code per se. That olive branch makes it very clear that MySQL has to focus on creating value through service – they cannot abuse their customers or community.

Lorenzo responds:

So, in summary, Russ Danner sees in open source the possibility for big business to take the work of various open source communities and use it to make huge profits. The communities themselves are, as we have seen, irrelevant. Any claims they make equally so (remember, licenses are irrelevant). So Russ sees open source as yet another way for the rich to bleed to poor. And that is good.

He goes on to say:

“I would be sorely upset if I found my development staff was hacking MySql code.”

I guess development staff means “wage slaves”. What does sorely upset mean? That you deny your developers the freedom to work on what they want to work on? Does it mean that the right to choose only exists for the capitalist class, the bosses? If one of your developers worked on MySQL code on his free time, would you punish him or her for that? What if the changes the developer made to MySQL was beneficial to Alfresco in some way, would that developer be credited for that? Or still punished for their insubordination against the elite way?

But could the open source movement work for social change? Well, not in isolation. But it can play a part. One must be careful not to see all open source users or open source developers as a part of a coherent whole. The only thing that connects us all is the fact that we work on or with software that fits the criteria of being called open source or free software. Otherwise you have the full spectrum of political beliefs, including Russ Danner’s capitalist ones, or the outright racist viewpoints of Eric S. Raymond.

Context is important here and I failed to provide that in what I said. If my developers want to work on MySQL or other open source software (that is contextual [rather than core] to my business) on their own time – more power to them. When it comes to what they work on at work? It’s important that what they work on serves the organization. I’m not saying every task must be direct. I spend a lot of time in the Alfresco community and yes.. my employer pays for it, and yes.. that is the right thing for them to do because its an investment in the employee (me) and the technologies they rely on (Alfresco.) That said… If we were to spend our time developing the core components of Alfresco we wouldn’t be serving the core needs of our business and that is wrong. We may be able to help and we are certainly willing but we do need to consider where we focus development. I believe Alfresco should look for community partnership in core development, but I also believe that it will be rare that they find it. Development is expensive business and both sides must find ways to leverage each other in a fashion, which is symbiotic.

I believe open source and business are important partners. Open Source and in general the ideas put forward in Clue Train bring balance to the system by empowering the members of the market with a voice and a recognized lion share of the power. It rightly positions the companies in a position of service rather than in a position of supreme power with the potential for the kind of abuse that is associated there. I have become a fan of commercial open source companies who use dual licensing and the GPL. It clearly states code or cash is the price for this software. This provides both the economic and social feedback effects to come to play more fully. Should people be compensated for their work? Yes, without a doubt in my mind. Should we have a social conscious? Yes Absolutely. I think that open source is capable of accomplishing both. Commercial interests do not have to be void of social interests. I believe in strong leaders who see social interests as commercial interests hence the Peter Drucker quote.

Lorenzo, thanks for your response. I will spend some more time thinking about what you have written.

Open Source - Growing Up

As far as I am concerned, OSBC is the most important conference of the year for those who are looking to use open source to create leverage for their business. If you were not able to attend this year … know that you missed out and you should make an effort to get to next years event; it’s the week of March 10th: plan accordingly. :-) For those that couldn’t attend, you can find a lot of analysis here on Matt Asay’s Blog, The Open Road. There were many interesting, insightful, and practical keynotes and sessions over the course of the two-day event. Of particular note: R0ml Leftowitz’s talk on IT: Forget The Factory, Enterprise IT is a Deli, all keynotes/footnote addresses, the panel on open source acquisition, and several of the panel discussion where customers were able to give real world feedback on their experience implementing Open Source. As a customer trying to walk in their shoes the customer centric sessions and keynotes are extremely helpful.

I am very grateful for OSBC, the folks who put it together and for those who take time to speak and attend – it’s simply an invaluable opportunity to learn from and interact with the brilliant folks who are changing the face of the software industry as we know it.

Every OSBC has had a transformative effect on my thinking concerning open source and this year is no exception. However, this year the thinking is of a much different nature than in years past and I’m still trying to digest and understand it. It’s probably not the best idea to blog until I have come to terms more fully with my thoughts but what the heck… what fun would that be?

This year I attended a talk titled “What can Open Source learn from Microsoft and the proprietary world.” Moderator Steven Walli (Consultant), Jean Barmash (Alfresco), Neelan Choski (SpringSource), Jim Zemlin (Linux Foundation) and Sam Ramji (Microsoft). I have to admit I was pretty excited for this talk. Steven Walli has a fantastic way of cutting through the spin and getting to the point. Sam is Director of Open Source and Linux Strategy at Microsoft. Jim is Executive Director at the Linux Foundation and Jean and Neelan are members of highly successful, fast growing commercial open source organizations. What a topic! What a line up!

Does Open Source have something to learn from Microsoft? My hypothesis? You better believe it! Microsoft’s business is orders of magnitude larger than the entire open source market as a whole at the moment. The open source industry is trying to institute open source from the bottom up. If Microsoft were to open up, it would have to do so from the top down. Small groups inside Microsoft may favor open source but real, timely change is going to take someone with a title that includes the words: “Director” or “EVP” and “open source” and a top-level management commitment. It takes control of the helm to effectively turn an aircraft carrier.

I’m not familiar with Sam Ramji (Microsoft Corp, Directory of Open Source and Linux Strategy) or his work – I’m not familiar with what kind of strategy he is responsible for concerning open source or Linux in his role at Microsoft. From the open source perspective, is his position used for good or evil? Is Sam a voice for change or is he a “Ring Wraith” carrying out the dark work Mt Doom. To be perfectly clear, I have no idea – I still don’t know for sure. I tried to attend the session with an open mind and with hopeful expectations that Microsoft might indeed be recognizing some advantages in opening up and thus they have someone like Sam to help champion and ultimately apply some of open source’s principals to improve their business.

Is shared source open source? I wouldn’t call it open source but they do have an OSI approved license. Does Microsoft’s brand of open satisfy the open source community?
From what I can tell: not at all. However, it is a start – at least that is how I view it.

Sam seemed to be coming from the perspective that Microsoft does in fact see some advantages in the open way. My conclusion? “Not a Ring Wraith.”

The panel discussion was good and all of the panelists gave strong presentations and good information. However, there was, in my opinion, minimal discussion on What Open Source can learn from Microsoft and the proprietary world. Instead, there was a lot of information, particularly from the open source side on how Open Source currently does it. It was all good information but I left disappointed.

I felt we had missed a golden opportunity to cross-examine Microsoft and the gentleman who seemed to be trying to facilitate some openness in the giant. We (Open Source) did a really good job of talking about and in some cases defending open source and a rather poor job of searching and listening. Sam’s perspective is different from “ours”, his problems are different and he’s working at the scale we all want to achieve (assuming you are a capitalist pig like me.) Shame on us. We should have made Sam work hard, really hard during this session. We don’t have to justify the existence of Open Source or the benefits of its models. The fact that Microsoft via Sam and Brad Smith were present at OSBC already demonstrates this for us. Further, do we honestly believe our current way of operating scales up to Microsoft-like size without modification – it hasn’t demonstrated that it can so far? Let’s not get lazy – even if we think we have the answers, we must continue to ask the questions over and over, again and again. “The answers change much more often than the questions.” -- Peter Drucker.

The open source community gets so worked up about how open Microsoft is or isn’t. I’d be willing to say we’re not going to be satisfied (if we even could be) with anything less than Microsoft’s full and unconditional surrender to our terms of openness.

What I think is interesting, and what I think you might find interesting if you really pay attention at an event like OSBC is that while everyone agrees there is tremendous value in Open Source, there is really no consensus on where exactly it is. Many point to code, community, licenses, cost savings, distribution etc. We understand to a large degree the overall effects of the system but we often get caught up in the details or are ignorant of how they work together to produce the effects that they do. What I’m saying is that even within the community many are still working to understand the dynamics of open. Given that, we’re not exactly in a strong position to say why it is that Microsoft or anyone else should be so open or just how open they should be. Opening up is almost a “personal” journey that has to be tailored to each organization that embarks on it. Openness is leverage but that has to be demonstrated and learned. Few are willing to take leaps of faith. Many are carrying so much legal, culture and other legacy baggage that the journey is slow, complicated and encumbered.

I'm not sure we are yet in a position to be telling anyone how it should be. We are in a position to demonstrate some real customer value -- but -- we clearly have more to learn and demonstrate. By our fruits not by our rhetoric..

Redhat is the crown jewel of Open Source but it’s only doing 500 Million a year. Compared to Microsoft, Redhat has some growing to do. Does Redhat want to scale to Microsoft size – I really hope so. I believe that open source delivers more value per dollar than the alternatives and at Microsoft’s scale that would constitute a huge amount of value. We’re never going to get there unless we think and ultimately operate at that scale – which means learning from those who are already there. It strikes me as sad and ironic that the open source community can sometimes be so closed-minded.

San Jose : Alfresco Community Summit

Today Alfresco held a community summit & bar camp in San Jose, Ca. In my opinion this was the most valuable Alfresco event to date. Great Job guys.

Unfortunately I was not able to attend in person but was able to catch most of the event on line.

The event was one highlight after another. All of the speakers did really well and every presentation had something to offer. To get the presentations log on to the Alfresco content community (http://hosted4.alfresco.com/alfresco/faces/jsp/login.jsp)

John Newton (Alfresco CTO and Chairman) began by describing at a high level, the vision for Alfresco, both as a technology and as a player in the market. Alfresco is working hard to build an open source based platform for Social Computing. Social Computing is all about collaboration via social media tools both in and outside the firewall. Social computing is a growing concern in the enterprise as new generations of workforce are hired on. Each generation is more comfortable with the internet and its ever growing brand of social tools (including wikis, blogs, and platforms for social networking like Facebook and MySpace) than the generation before it. These employees are used to these tools and expect them to be available regardless of where they are working from. IT on the other hand is often responsible for security and compliance / regulatory concerns. Many IT shops initially push back on these social media tools because they lack the enterprise features that meet the requirements mandated by the regulatory groups (SOX etc.) As we move forward in this brave new world where the lines between inside and outside the firewall are blurred we'll need a platform that delivers our social tools outside our corporate walls while maintaining our ability to meet our enterprise requirements.

John explained with great clarity that in today's market there are no products with exception to Microsoft Share Point and Alfresco that fill this space. Most other relevant vendors are far too specialized or are fragmented along diverse product lines. John explained that while Share Point is able to operate in the space, it is not an ideal platform for the job and that it also comes at the high price of end-to-end vendor lock-in with Microsoft technologies. Alfresco on the other hand has been designed from the ground up on open technology specifically to fill this space.

Additionally, John spoke about plans within Alfresco to increase the level of transparency and community involvement with respect to the product road map. Alfresco plans to improve on accuracy and management of customer expectations with regard to the content and timing of releases. Alfresco has decided that delivering on time is priority #1 with respect to releases. The saying goes "on time, on budget, or at quality; pick two." In this case, quality and completeness will take a back seat to on time delivery. Concerned?! Put your mind at ease -- this makes a lot of sense. Enterprises require predictability. An inability to plan can often be crippling to a project. Knowing exactly when the next release of Alfresco will be available is absolutely critical to a healthy relationship between Alfresco and "The enterprise." It's important to know that when the product ships it will have what you need in it. Alfresco is taking steps to increase this awareness as well. They have decided to publicly prioritize the work they are and the work they are not planning on doing. This system is known as the MoSCoW system. MoSCoW stands for Must do, Should do, Could do, and Won't do. This is a wonderful addition to the product/project roadmap. Community members will know exactly when the release will take place and what they can reasonably be able to expect to be in it. Similarly you will find a list of items Alfresco "Won't do." It is just as important to understand where we are not going as it is to understand where we plan to be in the future. I am looking forward to this new level of clarity and transparency in the the road map.


From a technical perspective, web scripts continue to take center stage. Jeff Potts, Optaros principal ECM architect and Alfresco's contributor of the year gave a brief overview of Webscripts. A Web Script is the ability in Alfresco to create a RESTful service in Alfresco. Building a webscript is a really powerful and easy to do. You don't have to be a hard core programmer to do amazing things with Webscripts. If you can handle a (very) little XML, javascript and freemarker you are all set. Javascript is a really light and very easy programming language used to describe business logic, Free Marker is a simple template languag, and XML is used to describe and "bind" the service to the system.

After Jeff's introduction we saw some fantastic examples from customers, service integrators and Alfresco employees of what can be accomplished with a simple webscripts containing a few lines of "code."

Also worth mentioning is that Alfresco is currently hosting a Webscripts competition. You can find out more about it here: http://www.alfresco.com/partners/programme/webscripts/
Everyone who submits a valid web script will receive a limited edition Alfresco T-shirt. Our first place winner will receive an Apple iPod 16 GB Touch, the winner will be announced on April 4th, 2008.

For me the apex of the Webscript wonderment was the demonstration of the Dynamic website framework developed in partnership with the Alfresco sales engineers, the community and the product development team. Michael Uzquiano of Alfresco led a entertaining tour of the framework, demonstrating how simple it was to edit a dynamic website driven by Alfresco. One of the best ways to make changes to a website is to find the page and component you need to edit on your site and simply edit it directly. The website framework allows you to do just that. The framework supports the ability to enter edit mode anywhere on the site, at which point you are able to see how the page has been partitioned in to components. From that point you can add, remove or edit the individual components. All of this works seamlessly with the rest of the Alfresco WCM capabilities including sandboxing and the ability to promote / deploy your website to different environments (staging, live etc.) This is really exciting work. Michael said that we can expect to see the work he demonstrated in the the approximately a month.

Again I want to thank everyone who made this event possible -- it really was terrific!

Open Source Gardening

Want to build an open source community? Roll up your sleeves and get dirty. Writing the code is tough but building an ecosystem is the work of the ages.

Sure some things happen naturally. In addition to making your source available you can put up forums, forges, blogs and wikis and people will use them but if you really want to see what kind of yield the garden is capable of you need a gardener or two. Gardening is hard but satisfying work – try it and you’ll see. My very first job as a young man was as a farm-hand on an arborer’s (tree) nursery – I’ve done a bit of “gardening” in my time. It was a valuable experience in my life; nurturing something, growing it, being able to see your hard labor prospering even years later. It is this same sense of making something grow that over time has caused me to gravitate away from technology and towards business. Code and other technology is by nature ephemeral. The system in which it takes shape and operates, the ecosystem is something more more enduring -- and frankly from a business sense, more important.

Alfresco has a few gardeners in the community – not all of them work for Alfresco, most do. A few of the engineers; Andy, MikeH, Gavin, Dave C, Kevin C have many many posts (over a thousand in two years.) The amount of time they spend caring for the community goes well beyond the work day. I can't thank them enough.

Nancy Garrity is Alfresco’s new Community Manager: Head Gardener in the Alfresco ecosystem. Nancy knows how to build and facilitate communities. If you’re looking to get involved in a hands on way, she is the one to contact (Nancy.Garrity@alfresco.com)

Lately I’ve noticed Nancy Garrity has been answering questions in the forums – and not softballs… tough questions. Is Nancy an engineer? No. So how is she involved? We’ll she is learning where the reference material is located and showing people were the can find it – and how they can help to improve it.

It’s important to put dedicated resources in to the community especially if Open Source is your business model. The ecosystem is your life-line. Alfresco gets it. With the help of Nancy and others like her we are proving that you don’t have to be a coder to participate. I’ve long though t that we need more non-technical folks in our community. We need to make sure Alfresco addresses real needs for real people.

What’s your take? How do we encourage non-technical, business users to engage the community? What should the conversation be?

Alfresco WCM 2.1.x review (on the eve of the 2.2 release)

Alfresco / Alfresco WCM 2.1.x has been out for some time and it's time for a review. I want to try and provide a balanced perspective. While given that I have an Alfresco blog -- I think the product is great there are areas that need work and I think they are worth sharing and discussing. That said:

Alfresco WCM is novel technology – there isn’t anything like the virtualization server, the AVM (Alfresco Versioning Machine), or the kind of link verification found in WCM out there on the market today.

The virtualization server is a preview server or staging server that allows all users to view the changes they have made in their individual sandboxes in the context of the content that has been pushed to preview. At first this does not appear particularly special in any way but it turns out the be something that really is quite special. Imagine two projects taking place on the same website simultaneously, during which massive CSS changes must be made by each team. Outside of WCM and its virtualization server you must set up two complete instances of the site and go through the trouble of keeping them in synch. The virtualization server however gracefully handles this issue by giving each team its own sandbox to work in. Each sandbox is a layer on top of preview, like a sheet of glass which superimposes the changes made in the sandbox while allowing the modified content of preview to “shine through” (the glass) enabling an in context preview of the changes in the sandbox. We’ll talk more about layers in just a moment. Before we move on it’s important to point out that the virtualization server doesn’t just virtualize standard web pages, it’s capable of virtualizing PHP, ASP and even entire J2EE Web applications and we’re not just talking JSP here – we’re talking servlets, application frameworks (struts, jsf) and inversion of control containers like spring.

The AVM is a content store that acts like the SVN source code repository on steroids. In addition to the SVN tree like capability which enables the user to version the enite site on each submit enabling the ability to time travel through changes/versions of the site, AVM introduces a concept called layers. Each layer is like its own SVN tree but layers can be stacked on top of or superimposed on each other to create complex views to better support collaborative workflows. The basic example can be seen in the Alfresco Web Client where layers are used to create a Staging or Preview area and individual sandboxes for each collaborator for a given web project (website, web project, etc)

The link verification is unlike other link verification in that it goes beyond the traditional static analysis found in most products to include server side includes and other runtime link checking.

That all sounds great – it sure does! Is there anything else? Yep – WCM has support for simple and complex workflows through JBPM integration, several deployment capabilities and a visual / GUI XML document editor (based on the XFORMS standard)

Anything else? Well there is a bit more in the product that just hasn’t been exposed yet and many more features on the way.

Version 2.1.x is the latest stable release and 2.9.x is the latest “Labs” release. If you are looking for more stability go with 2.1.x and if you are looking to try out the latest and greatest features available start up a labs instance. One of the best way to keep up with what is going on with WCM is to keep a watch on this wiki page: http://wiki.alfresco.com/wiki/New_Web_Content_Management_Plan

WCM has come a long way and in my opinion 2.1.x is the first release I feel is ready for prime time consideration. There are enough important features available to make the product a valuable tool. That said we have a ways to go in making the product truly great. Let’s talk about a couple of things that could use some work.

Roadmap: I’m a guy who likes to keep up with where things are going in Alfresco. I read the forums, I read the wiki, I download the source code almost every week just to see what’s shaking … the roadmap on the wiki doesn’t really help me. Road maps are hard – they really are but I am only being honest when I say I wish there was more there: a more distant look in to the future stretching from the certain to the very fuzzy -- with links leading to documentation and “white boarding” on the concepts. Personally I would like the tools I need to think about the software before I hear about it in a release scheduled, and before its been coded and is sitting in the labs tree. I want to be able to point out corner cases and use cases that are relevant to my needs and I want to plan 6 months to a year out based on how Alfresco is posturing. I may want to contribute design ideas and perhaps even code. The roadmap is the single most important community tool in an open source project – I don’t have proof for that – but I feel it in my gut.

Workflow: I can’t speak in depth about the workflow capabilities in WCM, I am simply not qualified to do so but I’ve spent a fair amount of time with working with the workflow engine. In many cases the out of the box workflows served up straight or with a few modifications will work for many of your needs. I tried to do a few things with the JBPM scripting engine like operate on a change set and found I needed to first write some java code. There are times when you need to put a workflow in quickly or patch an existing workflow. The JavaScript capabilities inside the workflow engine should be a godsend here but instead I found it difficult to implement.

Workflow Events: From what I can tell – and I could be wrong here, we’re still missing node level events. What is a node level event? An event is a trigger or set of triggers that are fired on create, update, and delete. Today you can fire a workflow off on a submit action but you cannot do the same when a user inserts a new document, deletes or updates an existing one. I want to kick off workflows based on content hitting the store -- not just when it's been submitted to preview. There are interesting semantics to this issue and it's not a clear cut as it is in the DM side of the house but I'd still like to see a discussion on it.

Web Forms: Web forms are very powerful and my personal thought is that as the product matures they will be one of the star features. Today they have a couple of issues.

• Web Forms: Web forms are implemented with XFORMS technology – a pending standard. But Web Forms aren’t described using XFORMS technology, they are described based on an XSD. That makes perfect sense – almost. The issue I see is it sidelines a standard for a proprietary approach and while it [the approach] is clever but it’s constraining. In order to make it work you need to follow the rules of the engine so your XML schema is constrained to what the engine will understand.
  
  
• Another point of interest is that the schema/approach model works great for simple models but falls flat on its face for complex forms. Schemas don’t carry enough metadata to render the kind of presentations required by forms of certain orders of complexity.

• Form generation is slow – even simple forms seem to have visible latency.

Search: Search in 2.1.x is something I’ve heard complaints on but haven’t hit the wall with personally. 2.2.x looks like it is making an attempt to address search capability needs.

Looks like a few issues … sure it’s not perfect but in my opinion, beginning with 2.1.x WCM is a very usable product with capabilities I think will eventually set a new standard in the industry's level of expectation for WCM products.

Public Service Use: Auditing

I've put out a couple feelers to get an explanation for the full intention of the PublicService annotation. In the mean time, I've been doing a little digging of my own. There are two places that the intentions for this annotation may show up: Explicitly in code and implicitly in policy.

Explicitly the annotation is being used to assist services such as the Audit Service.



The audit component implementation relies on public service identifier interface to find public service methods. The implementation for the Public Service Identifier checks for the annotation to make its identifications.

I'll update this post with more explicit uses as they show up.

Will there be any implied policy in the Alfresco SDLC (software development lifecycle?) I don't know -- we'll see what Alfresco has to say about that; this is a good question for Paul Holmes-Higgin or David Caruana.

Personally I find the notion of a public interface much more interesting if it means I can count on it in some fashion because being marked as public sets parameters on the interface in the software development lifecycle. I am much more interested what this notion of public service could mean in terms of the SDLC.

As a customer dependent on a non standards based (although open) API, I need to know what I can count on, what to expect; I want predictability and stability.

Public Services

If you are looking for an architectural representation of Alfresco, the best wiki link I know of is here. This documentation does a good job of putting Alfresco in context with the application server, client applications and breaking down the high level components.

Two of the most important pieces of documentation we need to maintain is A: A diagram and a strong description of what the current architecture and B: A well described road map of where we are going and how we plan to get there.

I try to do my best to keep up with how the architecture is evolving. The way I do it is by downloading the HEAD of the SVN source base, looking at the changes as they come down and comparing them to my current understanding. If there is an unknown or a mismatch, I head for documentation.

Recently we have seen a new addition in the source base (many actually) but there are few that caught my eye and one that I want to investigate immediately: an attribute placed on service classes called @publicservice. What a public service is isn't really clear to me yet. I've thrown together a diagram that kind of illustrates how I break out the architectural structure of Alfresco in my mind and how I am imagining public services -- what they really are remains to be seen. I suspect the diagram has some errors in it. Also with respect to the public services I see some services that I would expect to be public but that are not marked public. For example, the Tenant Service -- a new service that allows the repository to host multiple clients simultaneously and independently. This could be A: because I am wrong in what I suspect a public service is or B: because the concept isn't yet being applied by the entire team yet. C: Other :)



From an architectural perspective the notion of a "public service" could be very important to me. I've looked at many open source frameworks and often wondered why there was no distinction between public and internal code. How can you exist without this distinction? How can you possibly plan to refactor your code without putting customers at great risk? From my perspective a public layer gives the community a contract that says "It's safe to develop against this -- the life-cycle, the maturity, deprecation and ultimately removal of this interface is well known and you can plan against it." This creates a safety net for the community and a means or a protected space for refactoring for the engineers and architects of the project. If you the consumer have violated this public contract and tapped something below it -- it's at your own peril.

The documentation for "public service" doesn't yet exist from what I can tell and the java doc is somewhat uninformative in that it says "Specifically indicate that an interface defines a public service." -- a bit like defining a word by using the word. What does it mean to be a public service? I am not sure yet -- but I was thrilled to find it there and it has been fun to speculate on what the Alfresco team is intending for it. I plan to get to the bottom of the following questions:

• What is the definition of a public service -- can we get it in the java doc?
• Can it be counted on -- Is there some sort of contract with the community?
• If so is there a some sort of documented life-cycle?

I'll be back with answers once I get them! If you have the answers post them in the wiki and link it here!

Exciting stuff.

Freedom of the Press and Enterprise Content Management are IMPORTANT

Freedom of the Press matters dearly to humanity
Affordable Enterprise grade content management matters dearly to Freedom of the Press.

I think Alfresco, having gone completely to GPL last year has the opportunity to play a huge role in making the world a better place to live in by enabling the Freedom and ability to publish.

Related:

Sasa Vucinic invests in free press

and

Peter Gabriel fights injustice with video

Tired old vision of ECM?

Seth Gottlieb of Enter Content Here posted this blog on Alfresco, its new Facebook integration capability and what that demonstrates about Alfresco and it’s vision for ECM. Alfresco is integrating deeply with Facebook for social networking capabilities which from my perspective is consistent with their strategy of “oem”ing capabilities from the open source and open standards community that solve contextual problems and are best developed by engineers who are focused directly on that problem, for example, JBPM business process engine, MediaWiki engine integration, javascript engine integration etc

Seth remarks:

“Alfresco's strategy is to integrate with best of breed applications. You look at their website and they use MediaWiki for their wiki, WordPress for blogs, and Baynote for search. All the open standards and APIs give you great tools to do these integrations. I used to think of this as Alfresco talking the ECM talk but not walking the walk. Now I get that they see the old ECM vision as being as outdated as I do.”

Alfresco's implementation is based on more than a dozen complementary open source products. One of the attractive aspects of Alfresco as a company in my view has always been its discipline when it comes to focusing on core capabilities rather than getting bogged down creating proprietary implementations of contextual capabilities. The key to this strategy is making it happen in such a way that the architecture is consistent and the platform feels and acts like a single product. That is to say the functionality and APIs feel natural and related rather than bolted on. Following this strategy enables Alfresco to focus its innovation and engineering mindshare and energy on what is core to Alfresco: repository and ECM vision.

Early on this month at the first Alfresco community summit(s) (held in NYC and Paris, presentations here) we got our first taste of what Alfresco is focusing on as "core" both in terms of repository and in terms of its vision for enterprise grade content management.

On the repository front Alfresco is looking to continue improve on its enterprise capabilities:

• Scalable, fast, clusterable
• Federated view of repositories
• Web caching and clustering
• Multi-tenant hosting
• High availability
• Linear performance regardless of size
• Implementation of enterprise standards and infrastructure (BPM etc)
  
  
• Plug-in architecture

Also worth mentioning is the intention to consolidate the repository technologies that currently exist in the product. Alfresco DM (Document Management) and WCM (Web Content Management) are built on two different repository approaches. The plan is to define the superset of these two functionalities then harmonize and eventually homogenize the platform.

Alfresco brings a fresh perspective on the vision for Enterprise Content Management. Many ECM providers are focused on are focused only on the traditional ECM feature set: Collaboration, PBM, Portal, BI, Search. However, as Seth pointed out, Alfresco feels the future of ECM is in Enterprise Social Computing enabled by an open stack, open standards, enterprise grade high performance, and integration both inside and outside the firewall. What is Enterprise Social Computing and how is that different? The technology isn't all that different but mindset is entirely different. Where the technology does differ, it is in the fact that it is often "open", either the code is available or it is based on standards. The product's technical feature set is different in that in addition to the traditional ECM feature set, it includes capabilities for extending beyond it's own application space and beyond the firewall. However, the real change is in the frame of mind. It's a new focus on distribution, community, and collaboration over control. The power of transparency is evident in both the code and and this new vision of what ECM; what means, how we are supposed to interact with our content and with each other.

Welcome!

Welcome Alfresca readers! I've been participating in the Alfresco community since 2005 and blogging about it on my personal blog on and off since then. The size of the Alfresco community has really exploded in to something huge. Early this month we held two large user groups (one in NYC and the other in Paris), it was evident to me that it was time to start a blog dedicated to discussing Alfresco, the software and the community. I hope you will come back often, read and contribute!

As my first post, and because this is thanksgiving, I want to say thank you to the Alfresco team for putting together such an outstanding product with such a bright future! I also want to give a special thank you to all the engineers and community members who spend an incredible amount of time and energy in the forums helping the community... great job guys!

All, Welcome, have a wonderful holiday and stay tuned for upcoming posts!
-R