Keeping Open Source Open

Amazon recently teamed up with other companies to ensure Elastic search / ELK stack remains Open source. Back in 2018 Elastic started creeping in non-open source code. The fear in the community is Elastic will overnight switch the product to Enterprise and stop supporting the Open Source version.

Amazon recently teamed up with other companies to ensure Elastic search / ELK stack remains Open source. Back in 2018 Elastic started creeping in the non-open source code. The fear in the community is Elastic will overnight switch the product to Enterprise and stop supporting the Open Source version.

Episode Transcription


Welcome back to a new episode of The Byte. In this episode, we're going to be talking about Amazon's open source initiative and their latest blog article, "Keeping Open Source Open". And this whole article was kicked off because of recent trouble in the ecosystem with a company called Elastic. Elastic is responsible for Elasticsearch, so Elastic, Logstash, and Kibana, the ELK stack, and it's basically one of the central tools in the developer toolset. Now the issue is, as Elastic started developing this, they've been creeping in enterprise code into the open source space. And it's been difficult to basically see the difference between the open source and enterprise and it looks like it's going more enterprise-y. So there's a fear in the community that the product will just disappear.

Now Amazon, they've gone from being a very closed source company for a long time and now they're contributing more and more back to open source. I just checked their GitHub repo and they have, as of today, 134 repositories that they manage. And I was scrolling through and I noticed a lot of them are updated within the last few hours or a couple of days. So it's quite impressive that they are actually eating their own dog food, as they say. So they're contributing a lot back to the open source community. And in this article that they wrote, they said, whatever open source tools they consume, they want to make sure they contribute back to ensure that the ecosystem lives on and that the products live on as well. So I mean, they're part of the Apache Foundation, they sponsor KubeCon, and then some different conferences. So they're really involved in the community, so to say. Now Elasticsearch, what they'd been doing ... I mean, as I said, it's an essential tool for coders and in 2018 it was noticed in the community that non-open source code was creeping into the code base.

Now, Amazon and Elastic met together and they tried to work out a deal where maybe Amazon can dedicate resources to the project, really ensure that the project remained open. Because of Amazon customers, I mean a bunch of cloud customers was concerned about the state of Elasticsearch and the ELK stack. Because so many people are using it, if it goes non-open source, I mean, what happens? People probably will stay on it and we won't get any updates and the product will just eventually die. So what the deal was is Amazon teamed up with an Expedia, Netflix, and some other companies and created an open source distribution for Elasticsearch.

So you can go out there, it's a basically a fork of the code base that was unaltered before all the enterprise codes crept in. And I really see that everything will start going this way. So there's open distro for Elasticsearch. I mean, we'll start trying it here shortly, but they're putting a lot of effort into it. Amazon has resources, dedicated Netflix, and I mean, I wouldn't doubt that this becomes the new de facto standard. Because if it's 100% open source as they claim it to be, and Elasticsearch goes more enterprise, Elasticsearch might just eventually become an enterprise company and close off open source now that Amazon has taken over the project. So it'll be interesting to see how this hashes out. But I was very happy to see the backing that Amazon puts into open source, their commitment to the open source community, and also how they were able to get other companies involved in the project.

So I mean, Netflix is no small player. So they want to make sure that Elasticsearch is freely open for everyone and their users. So they are going to contribute a resource to it, Expedia ... and they want to make sure the licensing around open distro is all Apache 2 licensing. So it's very open, it's easy to reuse and re-support. And they're adding a lot of features to it already. I mean if you look into it, there's a lot of things that I was already impressed with. They've added some enterprise great features, they've put some nice active directory and open ID integrations, alerting, and they're, they're putting some more performance analyzers and stuff like this. So it's something to watch. As I said, we're going to try it out and we're going to probably put on a couple of customer sites just as a proof of concept to see if it works. And I'll keep everybody posted how it goes. If you want the links, it is opendistru.github.io. I'll put it in the show notes and I'll put the article "Keeping Open Source Open" in the show notes as well.
Brian Christner