Is REST really dead? What are the alternatives and how could they be so superior?
We’re joined this week by Peter Pavlovich, Principal Architect at EnerNOC, where we discuss the future of the reactive data.
If you want to stay up to date with the latest developments in the modern web development space, join us at the 2016 Angular Summit in Chicago. Save $300 with super-early-bird rates!
You’re listening to the “No Fluff, Just Stuff” podcast. The GABM conference series with dozens of dates around the country, they’ll be one near you. Check out our tour dates at NoFluffJustStuff.com.
Hey everybody, we’re here at the NFJS Angular Summit 2015, I’m joined with Peter Pavlovich.
Michael: You know, Peter, I’ll just kind of jump right in. You said something really interesting in the panel discussion the other night. If I’m to quote you directly, I think you said, “REST is dead.”
Peter: REST is dead, long live the Web. Absolutely true.
Michael: That got some murmurs in the audience. That seems like we’re at the Angular Summit, and this is this whole vision of this client-side NBC, we’re all getting our data from REST calls from the server, so what are you picturing? Tell me a little more about this.
Peter: So if you think about the reactive Web that we all live in right now, when was the last time you opened up your Gmail client and then hit the refresh button in order to see new mail show up? I can’t remember ever doing that. If you think of that as a situation where you’ve got a client and you’ve got a contract with that client, and your contract includes keeping their browser up to date without them hitting refresh, and I’m the one that’s got the server.
So I have a server, my client doesn’t have to hit refresh, and they’re happy with that, and I’m happy with that, they’re not my only customers. Right? If I also provide a data service, and I’m expecting my other clients who use that data service to have to poll to find out if something has changed, isn’t that the same thing? So why would I expect some of my customers to put up with having to push the refresh button while I don’t expect the other ones to, and in fact I am perfectly aligned with their demand that they not have to do that.
We have the technology, we have the design patterns, although we don’t probably acknowledge it as such, but we do have the technology to actually make that happen server to server now. There’s lots of different patterns that we can use to implement that, and we ran into this exact situation at the company I currently work at, which is EnerNOC, which is a green energy company, energy intelligent software company here in Boston.
We had that exact same problem that came up, where we were trying to integrate two different data sources merging into a single portal. The two servers needed to stay in synch with each other in order to present a unified vision of that data.
Michael: Data drift has always been a difficult problem.
Peter: It certainly has. By acknowledging that REST is not up to today’s requirements for server to server data communication if we are going to actually follow through with our promise to our end users that they will never have to hit the refresh button. In fact, they will be notified as quickly as humanly possible whenever data changes. Then we either have to be polling constantly, or…
Michael: Which is not sustainable.
Peter: Which is not sustainable, and also is still not real-time enough, because there always, your always going to have to throttle that to where instead of providing a REST interface, I would have to provide a callback mechanism where you will call me, you will say this is the data I want, this is the filters I want you to apply, here is the maximum time, the maximum rate at which I can expect you to call me.
I don’t want you to call me more than once every 30 seconds, but, I want you to call me at least every five minutes to let me know that you’re still alive, so I don’t have to ping you to find out whether you’re still alive. And, when anything changes that I’ve subscribed to, you will call that URL and you will let me know using an agreed upon schema for the data. That’s one way you can do it. Another way is that you open up a socket…
Michael: That’s the approach that I’m familiar with. So the only technology that I’m currently familiar with, that’s along these lines, is SignalR which was released I think a few years ago by Microsoft. It did a lot of these things, but it sounds like you’ve got even a bigger potential vision for that, especially going server to server, because it certainly really more down the Web sockets.
Peter: Absolutely, that’s definitely one way you can do it, or you can do it with a “We’ll call you, don’t call us,” sort of way. Another option is to use something like RethinkDB, which basically provides that kind of support out of the box, you could also use Meteor as your server that connects these things up, because it provides this DDP protocol which operates through sockets and you can have one Meteor server talk to another just with a single line of code, and it sets up this two-way databinding between the servers.
So as I said, the technology is there, there’s lots of different options for it. Really, there’s no excuse anymore for forcing people to hit the refresh button and that goes for whether you’re a Web client or whether you’re a server client.
Michael: It seems like there’s still some architectural hurdles to get past before this really becomes mainstream. But one of the things you said was the don’t call us we’ll call you approach, how do you see in broad strokes, how do you see that being implemented in a browser? Because you have so little control over the browser and then the surface area of that machine on the corporate LAN behind the corporate firewall, et cetera.
Peter: Sure, so I would not do that through a client, which is a browser. That would be an approach I would use for server to server communications. For browser I would use something more like Firebase, which does that, or Meteor which does that, or Rethink which does that.
Michael: So the server to server approach, I’ve always heard that termed as Webhooks or like an API push. So that’s a thing, and Web sockets are a thing.
Peter: They are.
Michael: And Firebase is a thing, so this is not a “my vision for APIs in five years,” this is we should be thinking about this right now, that’s kind of your point.
Peter: We should, and the only reason that we’re not is because it is still a hard problem to solve. If you’ve got a Postgres, or an Oracle, or a SQL Server database, and I’ve written my business objects and my persistence layer, I now have all of my clients’ code going back and forth, the data is flowing into my database just fine. The requirement is that I provide a data portal that people can call to get that data through an API.
The easiest solution is to put a REST API in front of that because it’s easy, I’ve already got the data objects behind that, that already are hooked up, the persistent service, I just have to have those APIs there in order to be a pass through into my existing business logic.
Michael: We have the scaffolding that generates these things now.
Peter: Absolutely, so that’s easy, breezy, not a problem, and it’s a single sprint task for a developer to do that. As opposed to saying, “OK, instead of that I now have to worry about whenever any data changes, I’ve got to examine any subscriptions which have been made by external clients.
I now have to filter all of those changes that have been made in the database, discover whether any of those subscriptions have been affected, and then package up that data in some sort of very efficient protocol, and then squirt that data out to those clients either calling each of their individual callbacks, or through sockets. Then of course you’re trying to deal with sticky sessions and all of that other problem to go with that. So it’s not actually an easy problem to solve.
We do have the technologies to solve the actual server to server communication, but determining what data to pass through those channels, that’s the tricky part right now. For instance, Meteor has solved that by tapping into the MongoDB op log, operational log. So they can now, they’ve been working with the MongoDB engineers to make sure that they’re able to efficiently parse that and figure out what needs to be passed back and forth.
I imagine that RethinkDB is going the same thing, the Meteor team’s actually working right now on a similar kind of thing for Postgres. They found some hooks that they can use for that, and I know that they intend to hit the other ones too, but it’s not quite there yet for some of those other SQL databases.
Michael: Is there a situation that you can see where a traditional REST would still be the right solution, or really is this I don’t want to use the word panacea, but this is the clear direction that things need to start moving?
Peter: If I were going to say that I was going to set up a new REST service, to me with the way I think, and being in the world of reactive Web sites, and reactive data sources, that all of us are entering with all of these new technologies, it would be to me as if you said, “Oh, well we’ll just put a refresh button the client screen on my iPhone.” Any UX engineer would laugh you out of the room if you tried to do that.
So we need to start thinking about our servers, our server to server communication, those are also our clients of our services. We have to say that what’s good for the goose is good for the gander, and if we’re willing to say to our clients, “No, we’re not going to force you to push the refresh button,” that should be consistent across all the interfaces that we provide data through.
Michael: Yeah. That makes a lot of sense. Peter, thank you so much.
Peter: Oh, thank you.
Michael: I’ll see you on the road soon.
Peter: You know it.
Michael: At No Fluff Just Stuff we bring the best technologists to you on a roadshow format. Early bird discounts are available for the 2016 season, check out the entire show lineup and tour dates at NoFluffJustStuff.com. I’m your host Michael Carducci, thanks for listening and stay subscribed.