“Diaspora and Me” — A reprint of Pistos’s address to the larger Diaspora community (Pistos Diaspora)

Posted: 10 March 2012 | Author: Building20 | Filed under: Diaspora* Alpha | Tags: dev, devs, diasp, diasp0raca, diaspora*, diasporapistos, opensource, pistos, podmin, podmins | Leave a comment »

(This was publicly posted on 9 March 2012 by pistos@diasp0ra.ca. I have reprinted it here in its entirety for posterity’s sake. To see the original post and the comments, click here: https://diasp0ra.ca/posts/218102.)

Hi. I just want to write here about everything I think and feel about Diaspora — or, at least, the more important thoughts and feelings. If I hadn’t made any friends on Diaspora, and nobody knew me, and I didn’t know anyone else, I probably wouldn’t bother with this. So, one of the reasons I’m writing this up is to serve as a collection of answers to what I anticipate will be some questions that will be frequently asked of me. Indeed, some of these things have already been asked of me, but I’ve been dodging the questions. Another reason for me writing this is that I’ve kept some things to my self, and I just want to get them off my chest. There is a burden I wish to unload. Although there is emotion in these words, I’ve still aimed to say only that which is worthwhile. I also want to get this out there to serve as a little bit of documented history. What I have in mind here is that saying “Those who cannot remember the past…” I don’t want what happened to me to happen to others.

tl;dr : I wish #diasporainc and I could work together. The future of the Diaspora project and me could be bright, or it could be bleak. I can’t tell which it will be.

Revisiting Diaspora

Like many others, I heard about Diaspora in 2010 when they published their video, announced their project, and started collecting funds to work on it. I didn’t do anything with or about Diaspora other than wait for my invitation request to be processed. Even after eventually being granted an account on joindiaspora.com, I did not seriously begin working with the code until October 2011.

Things went well. I contributed increasingly larger code patches, and they were getting accepted. The project team and I had discussions on github about my submissions, and we worked together to make any necessary changes to get my code merged into the main line of development.

But then, a trend started where some of my submissions would not get accepted. Sometimes, it would just be due to delay (i.e. it wasn’t outright rejection). But often, I simply would be left in the dark as to why my work was not getting merged. The primary example of this is Post Preview.

Impatience

At the time, I was just a user of joindiaspora.com, and I had to wait for them to deploy before being able to make use of whatever features or fixes I coded. Well, that didn’t last long. I soon put up my own pod, so I could push things to production as soon as I thought they were ready. This also gave me much more freedom to experiment with things. That is, in my own codebase, and not theirs. As I’ve said several times (elsewhere), in a multi-person, collaborative software project, I believe that it’s good for code to undergo review before being merged in. So, even though Diaspora Inc. granted me the power to push code to their repository any time I wanted, I did not exercise this privilege except for the most trivial changes, or when an external patch (or work of my own) demonstrably fixed a bug.

So, off I went, coding more things in my github fork (not to be mistaken for being a full project fork). I always worked in such a way that my patches would be mergeable back to the main line of development. I kept my fork up to date with their work.

Different Strokes for Different Folks

Over the month or two that followed, I began to see things about the way Diaspora Inc. was managing the project that I didn’t agree with, or would do differently. They may have (or have had) good reasons for doing whatever they did (and do), but, by now, I no longer have much desire to determine what those reasons are. They have the right, of course, to run their ship however they want. But I will just enumerate some of the differences they and I have, because people have asked, and will probably continue to ask.

Commercial Involvement: As I’ve expressed repeatedly, I do not trust Google or Facebook with any information about me. The same sentiment applies to any similar large company. So, I do not want to be on a pod that uses these companies for anything. joindiaspora.com uses Google for analytics and Amazon for static asset storage and serving. Diaspora Inc. also uses Google Groups for their mailing lists. Google’s CDN (content distribution network) is used for some javascript files (removal of that on my own pod is pending).

Pod Neutrality: I believe any primary, popular or main fork of the Diaspora codebase should not have pod-specific code in it. Even as I write this, many things in Diaspora Inc.’s codebase have to do with joindiaspora.com specifically. In contrast, I believe that the main trunk of development should be utterly pod-neutral, and any pod-specific code should reside in a separate branch, or should be added on in production by way of a modular interface. Otherwise, other podmins have to undo or remove the joindiaspora.com-specific things before deploying to production. I don’t believe any pod, not even JD, deserves to have any kind of elevated status in the codebase. In my fork, all diasp0ra.ca-specific code (landing page, footer links, privacy info page, etc.) is NOT in the main line of development.

Stable Branch: I believe that some branch (or series of tags) in the code needs to be kept as stable and production-ready as possible. It could be the master branch, but it doesn’t have to be.

Megapods vs. Micropods: I believe that improving single-pod scalability is not as important right now as other things that deserve developer attention. Primarily this is because the “classic” code (November 2011) ran acceptably on small- and medium-sized pods. It only ran into problems at much larger scales (hundreds of everyday-active users). Instead of working at scaling user count, I think the focus should be at scaling pod count. That means two things: Easy, secure and comprehensive account migration; and easy pod installation, upgrade and maintenance.

Money-driven: While the members of Diaspora Inc. are free to pursue their living how they like, I myself don’t need Diaspora to make money, so I work on Diaspora without seeking remuneration. This allows me to be free to set project priorities as I think they should be, and not be subject to the influence of the bottom line, any board of directors, or needing to keep a company afloat or alive.

Communication: The most critical difference between Diaspora Inc. and I has to do with communication. I believe that they do not communicate enough. Please note that I did not say they do not communicate at all. I’m saying that I think they do not communicate enough. Communicating with them is like having a phone conversation with someone who responds only to every fifth sentence of yours. Or writing to someone who only responds to one in ten emails. Or dating someone that stands you up every fourth date, and doesn’t return your phone messages. In short: It doesn’t work.

In contrast, here’s how I run my projects: I keep my IRC (chat) client open 24/7/365 in a support channel. This lets me see every support request, and I respond if I happen to be around when people ask questions. All my projects are on github, and I respond to comments, private messages and pull requests there. With respect to Diaspora in particular, I follow relevant hashtags and comment on posts that are asking questions, reporting bugs, or making suggestions. If I am particularly busy, and don’t have time to provide an in-depth answer to something, I almost always respond anyway, with a short answer, or an apology that I’m busy, and that I’ll get back to the person. I make use of issue trackers, wikis and websites to disseminate information. I make use of rubydoc sites to provide code-level documentation. If certain questions come up frequently, I take the time to write up answers, and publish them in a FAQ. I give communication a higher priority than coding. Yet, I still manage to code.

Some might point out that there’s a difference in size between Diaspora and my projects. This is true. But that doesn’t excuse bad communication. As your project scales up, so will the ease with which you can recruit people to help, and that includes helping with communication, whether by way of documentation, helping people in the support channels, or doing general public relations work. Good documentation also can relieve the need to directly communicate with people too often.

The Rift Begins

I tried for quite some time to maintain merge compatibility with Diaspora Inc.’s code. Around the beginning of December 2011, though, their main development branch became too unstable by my standards. At that time, I still wanted to stay compatible, so my attitude was that I hoped that their master branch would stabilize soon, so I could continue merging their work in. Unfortunately, that basically never happened.

In early January 2012, they released a major change to the code. This caused such severe divergence between their codebase and mine (and others) that it was practically unfeasible to merge work in either direction. For any of the work I had done up to that point to have had any chance of getting merged in, I would have had to rewrite it against their new codebase tip. It was a very discouraging turn of events. There were two things they could have done that would have made things better, but they did neither.

First, they could have temporarily closed the door on pull requests and then dealt with all the outstanding pull requests prior to embarking on the major change they implemented. Instead, basically all outstanding pull requests (not just mine) were rendered unusable. All the person-hours that went into them were garbaged. This upset a number of developers, not just me.

Second, they could have communicated their intent with the codebase before starting any work. That way, people would at least have known that they should not try to branch anything off the existing master tip, to avoid wasting time and effort on something that would need to be redone when Diaspora Inc. was finished.

With all these differences of opinion, the major code divergence, and a really feeble level of communication, it became clear that I could no longer try to work with Diaspora Inc. I had to fork my codebase — these factors made me. I didn’t want to fork. I knew very well that having significantly divergent codebases would make future merges difficult or impossible. Later work in one fork would have to be rewritten to be used in the other fork. Despite these realities, I went ahead, because it was either fork or stop working on Diaspora altogether.

What Now?

In late February 2012, Diaspora Inc. made plans to change federation and the inter-pod protocol. In all likelihood, any codebases using the old protocol would become incompatible, and, therefore, any pods running those codebases would be cut off from the network of pods running the #diasporainc code. This also means the Friendica network would be unable to communicate with Diaspora (until and unless they made all necessary charges). There are rumours that documentation of the changes will be released, but I have my doubts. Work has already begun on the new protocol, with zero documentation shown or discussion had about it (in public).

I have had mixed feelings about Diaspora for some weeks now (it is early March 2012 as I write this). I only bothered to continue working separately on Diaspora because I assumed pods running my code would continue to remain compatible with the greater Diasporanet. It never occurred to me that there would actually be changes that would cut off our pods. With the advent of these latest events, the continued lives of my fork and pods running it are in serious jeopardy. I suppose it hinges on how easy they make it to implement the new protocol (in other codebases, and in third-party networks like Friendica).

But how long can my divergent codebase stay alive? It hasn’t happened yet, but one day they’re going to come up with a killer feature that will be difficult or impossible to code into my fork. That would be the beginning of the end.

On a technical level, I would only work off their codebase again if it were consistently stable, and it had basically all the features that regressed since December 2011. It’s March 2012, and they still haven’t done that (comment Like and working line breaks, anyone?).

But, more importantly, on a personal level, I don’t want to work with them because of how they are. Not how they are as coders (technical), nor as acquaintances (social), but as colleagues (professional). Sean Tilley embodies the final hope I have in Diaspora Inc. because he actually communicates. And I don’t just mean with me. I see him talking with many others, and there’s probably even more he does that I don’t see. Please thank Sean the next time you run into him. The single most important thing Diaspora Inc. has done in 2012 is hire Sean, because, in doing so, they are addressing the issue(s!) of communication.

I intend to continue working on Diaspora and serving the Diaspora community for as long as my efforts bear fruit in the Diasporaverse. However, if the actions of #diasporainc begin choking my fork, that is not a battle I can win alone. We shall see what the future holds, my friends.

#diaspora #diasporapistos #diasp0raca #pistos #dev #devs #podmin #podmins #opensource

Specific Q&A

Here are answers to some specific questions asked by people on this post.

What makes you work so hard to develop and administrate this site?

I believe in some of the ideals that the Diaspora project espoused at its inception. Non-commerciality, privacy, decentralization, open source, and more. I want to be a part of the change I want to see in the world. I’m (usually) not one to stand idly by when something can be done, needs to be done. I also get satisfaction from coding things that people ask for and want. It’s gratifying to make something easier for people, or cooler, or take less time, or be less frustrating.

Do you see the pistoscode as a fork, or just a temporary state while the code is waiting to be pulled into the main codebase?

It used to be a temporary state, but by now, it’s pretty much a full-on fork. The only things missing might be a different name, marketing, and a competitive spirit. I’m confident that the code I’ve written up to this point will never enter #diasporainc’s codebase, for at least two reasons: Our codebases are too divergent right now; I haven’t written it to their specifications (e.g. with respect to testing). I have always been willing to write to their standards (technical obstacle), but I am not willing to work with them for as long as they continue to behave the way they do (social obstacle).

Where would you like to see Diaspora and you go, moving forward?

I would like not to have a fork (and, in fact, not to have to administer a pod). I would like to see #diasporainc make all necessary communication-related improvements, stabilize their code, and repair all regressions that occurred since the beginning of 2012. That would be a good start. I would like to work together with them, because they understand a lot more of the code than I do. It would be very helpful to get their assistance in understanding certain parts of it. Plus, more people rowing in the same direction is better than fewer.

However, I don’t consider there to be any realistic chance of these things happening or changing.

Does forking have to be a bad thing?

Forking a project is generally done out of necessity. A concession. A choice of an ill to avoid a greater ill.

Is it possible to organize the code so that there is the vanilla version (the core dev’s version) and then augmented versions such as the pistoscode? (So people can pick and choose what they want to use for their pod installations – kind of like Drupal?)

It is definitely possible to have a baseline, pod-neutral codebase, and then a modular and/or configurable system that allows podmins to choose features. But that’s not how #diasporainc wants it to be (as I’ve mentioned above). It isn’t even about forks or not. Even if there were no forks, this is still a principle worth adhering to.

How do you find the various representations of the core devs as has been posted on the stream?

Naturally, it’s not possible for me to have seen everything, but I will say this: I still give each of them the benefit of the doubt as far as outright greed or malice goes. I think people should not pass conjecture off as fact, and we should also try not to phrase ourselves such that there’s risk that speculation would be mistaken for certain knowledge.

What bridges do you think are important to maintain in the community? How would you like to see that happen?

There are two important lines of communication that should be kept healthy and vibrant: that between a project and its users, and that between a project and its contributors. How could that happen? That’s for #diasporainc to answer. I walk the talk with my own fork and pod. Refer to what I said above about communication.

What is your vision for a decentralized, privacy sensitive social network? (i.e. the software)

I actually have never taken the time to formulate a high-level vision for Diaspora. Up to now, I’ve just wanted to dialogue with the users, and work on some of the things they ask for, or the things that I myself want to see in Diaspora (as a fellow user).

For starters, I think, if anything, I would want a social network that had built-in immunity from commercial interests, even if only to a certain extent. No data mining, no tracking, no ads. Not influenced by greed, and resistant to any central control. I would want a social network to run on software that was free (liberated), so that it could be freely shared and improved by the global village. Going beyond that, I would love to see the social network run on non-commercial infrastructure. I don’t know how that could work right now, but imagine if your social network ran on, say, a wireless grid of easily-affordable machines, strewn all across your neighbourhood, your city, your country, with no involvement of any ISP, cable company, phone company, server company, government — none of those things. Nothing but hoi polloi, the people, the masses, doing their thing, by themselves, for themselves.

What is your vision for the social aspects of the D* community? (i.e., the community of coders and users as a whole)

With strong communication, so much is possible. Almost everything follows from that. Stifle that, and you are planting seeds in the desert. Every effort becomes undermined; stunted and crippled to be less than what it could have been.

What do you foresee in the near future for diasp0ra.ca, relating to federating with the main pod, etc, in light of their changes? What do you plan to do about federation?

If they make it easy enough, I will adjust the code to be able to communicate with the greater Diasporanet. If it’s too hard, we’ll have to assess how viable the smaller network will be, whether it would be worth keeping alive. I have no inclination to reverse engineer anything, or to decipher uncommented source code.

Will you be throwing away the baby with the bathwater and removing the old protocols entirely? More generally, will pistos-diaspora in the medium-long term federate only with Diaspora Mainline, or with everything else too?

I hadn’t considered maintaining backward compatibility. If it’s not too much effort, or if there is significant demand, I will, but we’ll have to see. I’m not that interested in doing much work as far as external network compatibility goes, in the interest of limiting scope (workload).

Do you have any plans to continuing to develop the API so we can fix it?

I like what people have been doing with the API so far, and the API is actually one of the easier things to work with in the code, so I think the API will enjoy a lot of attention within my fork.

Would you continue working on diaspora-pistos if the jd team decides to pursue mega-pods and database-level federation instead of old-fashioned pod to pod federation?

If things change on #diasporainc’s side, and people still want the “classic” Diaspora that my fork offers, then I would continue to work on it, and support podmins, and dialogue with users, and so forth. So, I would leave it up to the people to decide.

Do you think of asking publically for help in development? i cant imagine how far diasp0raca could go if you had some help…

If I did that, I think that would cross a line: I would first have to decide whether I want to directly compete with #diasporainc. I am not willing to take that step right now. Of course I welcome any help, but right now, I will not openly advertise for help, nor approach known contributors to ask for any committed, regular help.

Would you accept donations? Where can I donate some money for you to start some funding like geraspora?

I do have interest in accepting financial assistance to help with the server costs. Unfortunately, I have not found any payment system which is available for Canadian recipients and which suitably protects my information and identity. As such, donating is only an option for Canadian donors who can send Interac Email Transfers (a fast, safe, anonymous payment system which is only available via major Canadian banks).

---