WebUI Discussion

Thursday, Medina, 14:30

Convener

Joseph Holsten

Participants

Davida, Jordan, Jon, Nathan Smith, Charles Johnson, Jimmy Diamond, Brian Scott, Courtney, Noah K., Sean

Summary of Discussions

"So you basically have the whole front-end team at your disposal."

Nathan: "I maintain the WebUI, and it's for viewing your data, and you can peruse everything. But if everybody says that they don't use the WebUI, maybe we're not building the right thing?"

Joseph: "You're not, but that's life. Never building the right thing. "

Nathan: "I'd rather people say 'I always use the webui that does all these things that knife can't do.'

Brian: "It's great for consumers of the Chef data."

Courtney: "The AWS console is for people who aren't power users. It's for conversion of upper management."

Charles: "It closes us hella deals."

Nathan: "It's great for visualizing Chef concepts too, helps people understand quicker."

Joseph: "I've forked the open source webui into just a JSON API. I mean, everyone writes a custom dashboard"

Noah: "What's this add that normal Chef API doesn't have?"

Joseph: "Is there a good Chef implementation of Chef auth?"

Nathan: "So here- for analytics we are doing client-side oauth2 with ocid, so if you had the chef Server APi that supported cores and some authorization like oauth even though oauth is terrible that you could do client-side, THEN you could make an in-browser client that talks directly to a chef Server. That would be SUPER cool."

Joseph: "That's what I want."

Noah: "ChefWM should support auth other than keys."

Nathan: "The ruby side of Chef webUI is translate as an auth proxy. Server needs a token auth mode. See the thing that everybody uses is oauth2, and - 'My biggest fear working at Chef is someone will say Nathan, the board wants to talk to you! They need you to explain oauth2 in deep technical detail! NOOOOO!'"

Noah: "I wouldn't say erchef should support oath2, it should accept token auth. Then we can request a token, using key auth, and use that token in the future."

Nathan: "In OCID we have a part where if you have a key and you make a request asking for a token, it'll givey ou a token."

Noah: "That should just be in erchef."

(A WILD PIVOT APPEARS)

Brian: "What about making the WebUI read-only as a global policy? That would be really useful."

(Implementation quibbles ensue).

"CSS color:Grey over everything..."

Consensus: Doable.

(Back on track) Joseph: "So once token auth is in, then people can write their own UI."

Noah: "Then we could have a gem that sets knife token create, put the token into a dashboard, and then..."

(Bikesheds get painted)

Brian: "Our server has a hack in it that lets us auto-add users to an org every time it gets created."

Noah: "There is a special magic key that lets you pretend to be anyone else. Used almost exclusively for the webui, because it has to be trusted and impersonate other people. WebUI pretends to be you, and that impersonation key is the biggest impediment to custom UIs."

(Worried nodding silence.)

Joseph: "So this is why oauth is complicated. This shed has a mailing list."

Noah: "OCID uses the impersonation key. It has to."

Joseph: "So we don't wanna have the oauth conversation right now. So what else? You guys think that regular users don't use the UI so that implies something is wrong with it. That it isn't doing enough."

Nathan: "The CRUD for Chef objects thing, that's useful. Great demo. But there are a lot of other use cases that are stuff like what Joseph & Brian have built that are things that - you can't really visualize certain things easily through a commandline. I'd like web thingies that let you do that, advanced Chef users could say I use this all the time!"

Noah: "I love the run list expansion in the webui. It's so good."

Joseph: "I drill into web ui for node data.

"So there's always CRUD operations, done through git repo sync. Then there's the read-only version. Can you just show me the list of servers with this role? I need to know it's not on fire."

Joseph: "So always point in time data. What about that?"

Analytics!

Nathan: "Analytics is another product with its own UI and reporting will go over there. Gets cross-referenced into manage with links, but it's its own tool.

Sorry Noah, I didn't mean to talk about data bags in front of you."

Charles explains Analytics: "Chef Server is a point-in-time snapshot of your infrastructure right now. Analytics is a time-series database of those point-in-time changes."

Joseph: "So as an OSS user, the only thing we really had to see the history on was the git repo. How roles changed, how data bags were changing, etc. Seeing how nodes changed over time would be extremely valuable."

Charles: "That's why it costs money."

Joseph: "Seeing resource changes over time would be really useful too. Brian, your dashboard has this, right?"

Brian: "We've got resource changes over time, roles over time, and we're about to have versioned roles over time."

Noah: "Datadog is really good for that."

Joseph: "Neat!"

Nathan: "Everything in analytics has APIs for all this stuff, and you can shove data into Datadog or whatever. What's going to get public are the schemas - it emits messages from erchef when things happen, and the schema of those messages will be public, so if you wanted to build your own analytics using the same data format you could, or you could just pay us."

"So management console is going to be CRUD for objects on the Chef Server. A lot of the stuff we've talked about for cross-referencing objects should go in. People also want to launch cloud instances through it, but then we're like"

Brian: "What if the UI could support widgets?"

Noah: "I was promised that would never happen."

(laughs)

Joseph: "I built the cloud instance log - there's no real state information that's polled to the UI. You gotta go look at syslog from the boxes as they're building."

Nathan: "Chef reporting can show you runs at the end of the run."

Joseph: "But not polling rackspace for instance state while it's launching - blah blah blah. Anyway I built that, but I don't want people to use it."

Noah: "That gets into building other ui's using similar tech. Chef WebUI has a nice backend but it's not pluggable. Token auth and OCID together give you a nice solution for that."

Joseph: "Problem I had was - I have universal AWS console that works for 10 datacenters, and I have my Chef Server, but having a single-button build an instance and deploy that works in many datacenters is hard because that UI either has to cache a shit-ton to know all the things"

Brian: "We have sidekick running, polls all the cloud providers. That state you're talking about is polled every so often by Sidekick."

Joseph: "You're using an intermediate store"

Nathan: "Redis?"

Yeah.

Joseph: "I can get that information by making two calls and interleaving for hundreds or thousands of nodes - pagination nightmare - or I can do it server-side that is impossible to widgetize because if you're zipping together different data sources, how do your widgets do that? So that was my problem with the pluggable webui. Every idea I came up with required this coupling. Wasn't so much that you couldn't have them in different data sources, just wasn't performant at a UI level if we didn't have some pre-processing and a cache that required the widgets to be aware of one another."

Nathan: "Rails backend for the current webui does some stuff that's"

Noah: "It's unfortunate."

Nathan: "The reason it does that is there were things that people wanted in the webui and Noah went to build them and said well, the API doesn't do this. So there was some 'don't use the api but you still have to do it.' "

Noah: "Did you just call my code insane?"

Joseph: "As someone who re-implemented what you did, I'd be shocked if it weren't insane."

"So any ideas on how to build a widget API? Get data from my time-series data, other sources, and aggregate it."

Noah: "OCID and token auth gets you that. OCID gives you a token, from there you can just make ajax requests with the token and go to town."

Nathan: "You could build a proxy layer or just use a bunch of iFrames."

Brian: "Is this a product that Chef should be making?"

Noah: "Product I want is just a token auth. I can build the rest."

(Another bikeshed gets painted.)

Nathan: "We're doing client-side auth with analytics against OCID. Analytics API should support token auth mode out of the box, so if you do knife raw myserver/id/oauth/grant it'll give you back a token. Already in analytics API. Precedent to put it everywhere."

Features CHEF could crib from Brian?

• A query builder. Being able to search & report Being able report on any node attribute - "give me all the boxes that have a particular package installed in these data centers."
• A diagram builder. Show role / cookbook / node relationships. Show network relationships.
• Health inspector in the UI. Specify sources and see which code is not synced to the server.

Brian: "There's some undocumented keywords in metadata.rb that you might be able to use to generate these relationships."

Sean: "I'd like to talk about - our ops team loves the standard OSS Webui. So the question is, will that webui work with Server 12?"

Nathan: "No."

Noah: "If you point it at a single org..."

Nathan: "OK. Yeah. In current state it won't out of the box, because the user auth endpoint is uses - it takes user and password in OSS, and ent it takes username + pass, and the json it returns is slightly different. So you'd have to change that, and then"

Noah: "Also you'd need to point it at a single org."

Nathan: "Also it doesn't support multi-org."

Sean: "So what happens when it goes away?"

Noah: "It goes away. Rails security updates for another year, and then... that's it."

What will we do now? What needs to happen next?

• CORS headers in Chef 12 before final release.
• Need an RFC for a token auth mode in Chef Server / OCID.
• Chef WebUI needs a read-only mode flag (feature request)
• Query builder for Chef WebUI as a feature request.
• knife UI discussion should happen tomorrow.