➿ Jonathan and Josh iterate through the Go 1.24 changes and related proposals
This show is supported by you, our listener. Stick around till after the news for more on that. This is Cup of Goat for December 20, 2024. Keep up to date with the important happenings in the Go community in about 15 minutes per week. I'm Jonathan Hall.
Jonathan Hall:And I'm Josh Bleecher Snyder. Hi, Shy. Wait a minute. Just tell me what you said.
Josh Bleecher Snyder:I'm totally Shy. I have run my voice through an AI, adapter and also lost all the useful knowledge in my brain.
Jonathan Hall:Welcome, Josh. It's good to see you again.
Josh Bleecher Snyder:Likewise.
Jonathan Hall:It's been a while since you've been on the show. Before we jump into the news, let's do a quick overview of what's coming up in this episode. We have some releases, official releases, conference coming up, and we're mostly gonna be talking today about some of the new features coming in 1.24. So if that's your cup of tea or go, then, stick around. With that, Josh, why don't you kick us off with the most recent official release?
Josh Bleecher Snyder:Go 124rc1, the first release candidate is out. Woo hoo. So you should go and grab it and try it out. Google has said publicly that they run the release candidates internally. They run their software on it.
Josh Bleecher Snyder:So though it's not a full production release, it should be good enough for you to try, and this is the most important moment for your feedback. This is the last chance we have to fix things in Go before they're set in stone forever. So please do, at the very least, run your tests with it.
Jonathan Hall:Good advice. Also, in the official releases category, we have a security release for the golang.org/x/netmodule, specifically the HTML package in there. There's a potential DOS. Certain formations of HTML could cause resource exhaustion, if I could speak. So if you do anything with HTML and it doesn't really, if
Josh Bleecher Snyder:you do parsing it with a regular expression, for example?
Jonathan Hall:Oh, that's that's always fun. I love parsing, anything and everything with regular expressions. I learned that from Pearl. Alright. One other quick update.
Jonathan Hall:This isn't an official thing. Like like like, maybe it is, I guess, but it's not released. I think we talked about this on the show a month or 2 ago, but, just another reminder, the end of next spot. So that's January. January 22nd to 24th, GopherCon Singapore is coming to, well, Singapore.
Jonathan Hall:So if you're in that area, tickets are still available. Check that out, and, hopefully, we'll have some videos, later on we can share about the things we're spoken about there.
Josh Bleecher Snyder:And with that, on to the 124 release notes. Working our way through in no particular order, I wanna call out some additions to the strings and bytes package. This is new iterator support for iterating over parts of a string or a byte slice. The most exciting one is lines. Strings dot lines, you pass it a string, and it will give you an iterator that yields the lines in that string one at a time.
Josh Bleecher Snyder:It's the thing that we all used to do with, buff.io.scanner. Now it's a lot simpler, and you don't have to think about, like, is this a thing that can check errors or not? There is also split seek because I guess split was already taken, which is, strings dot split, but as an iterator, and fields seek, s e q, like sequence, which is like strings dot fields, but as an iterator. There are a few other minor ones, split after seek and fields func seek. These are really exciting because they're things that I used to do all the time in an awkward way.
Josh Bleecher Snyder:And to my mind, one of the questions I always ask myself for how excited I am about a new API is whether I backport it. And I have taken all of these things, stolen them from the 120 source code and 124 source code, dragged it back into my own project so I could use them now. So I can tell you, they're wonderful, and there's something to look forward to.
Jonathan Hall:Awesome. Yes. That line just gonna save me a bunch of lines of code, I could tell you. Every time I have to use, Buff. Io for that, I always have to go look up how to do it again, you know, what what's the exact syntax and what's the exact, you know, way that I'll I can forget everything I know about Buff IO now and just use this.
Josh Bleecher Snyder:And it's beautiful that they all share the same output because one of the ways that I use these was in a streaming parser where I wanted to break up the input in a bunch of different ways and test to make sure that I got the same output. And now I could drop all these in in a table driven test to be like, give them to me line by line, give them to me as fields, give them to me split by spaces, etcetera. Yeah. Very exciting.
Jonathan Hall:Yeah. I was gonna ask, are there, utilities to do this sort of like we already have you can range over a string and you get the bytes for it. Right? Are there iterators for that too, or do they figure that's already built in, it's good enough? Not?
Josh Bleecher Snyder:I think that already built in and good enough. I did write my own, a byte wise and a runewise, and they're, like, 5 lines of code. So if you need them, you can just, slap them in, or maybe, somebody will go and file a proposal to put them there alongside the rest. We'll talk more in a few minutes about one of the challenges between iterators and things that already have iterator like properties. So, put a pin in that.
Josh Bleecher Snyder:We'll come back to it.
Jonathan Hall:Okay. Sounds good. What's next?
Josh Bleecher Snyder:Also, in the 124 release notes, sync dot map has significant performance improvements. So I was around, that's how old I am, when the first sync dot map was being put in. And there was broad agreement about what the API should look like, that we wanted it to be a concurrency safe map that you could do atomic like things with. And then the question was, how do you implement that? And we settled on the dumbest possible implementation, which is like a copy on write with a new text.
Josh Bleecher Snyder:And we figure, okay, the main use cases that will become stable over time, and then all of the reads won't conflict with each other, and it'll be fine. And that lasted a long time, and then it turned out some people in fact wanted more out of it than that. So the big change in 124 is instead of having one single map that shares all the information, which you could then have a lot of conflicts on if you're doing a lot of updates, The map is broken up into lots of little pieces. And so now if you're modifying one of the pieces of the map, you don't necessarily have to block reads from other parts of it. So if you have a write heavy use of sync dot map, your performance should get a lot better.
Jonathan Hall:That's cool. Yeah. Happy to hear that. I don't think that's one I use very frequently, so it probably won't affect me, but I'm sure it will be a lifesaver for many people listening.
Josh Bleecher Snyder:Yes. It's one of those, rare but high pain scenarios. So one person out there is like, yes, yes, yes, yes, yes, and the rest of us can shrug.
Jonathan Hall:So speaking of things where people say yes, yes, yes, yes, yes. I am super excited about this new, this next one we're talking about. We actually talked about this in the show, I don't know, probably 6 months ago when it was a proposal. But the 1.24 will include the context method on the testing. Ttype.
Jonathan Hall:So I don't I mean, I we we talked in the show in the past about some of the use cases for this. I think the biggest use case for this, though, isn't per se practical. It's just a mental relief. Whenever I'm writing a test and I need to pass in the context, I no longer have to think. Should I pass in context dot to do or context dot background or something else?
Jonathan Hall:I could just pass in t dot context, and it will be the right thing in most cases.
Josh Bleecher Snyder:Absolutely. That is that is my favorite thing is not having to think anymore. I will say I've gotten in trouble in the past not having to think about contexts because you always just pass in whatever context you have handy, and then you forget that, like, you're doing a database right, and the context you have handy is an HTTP request context, and you realize that the, if the user hangs up, the database right is gonna get canceled midway through because you use the request context, and you actually do have to use the background context. So I've been in trouble before, but for the most part, just using t dot context and not thinking about it is absolutely gonna be the right answer. One other little funny story here is that there was so much talk about adding it to testing dot t.
Josh Bleecher Snyder:Nobody forgot about testing dot b for benchmarks and f for fuzzing, And there was a last minute discovery well into the r c one that, oops, we need those 2. So they've been added.
Jonathan Hall:Well, I'm glad they got that. So the
Josh Bleecher Snyder:next thing I wanna talk about is pair of proposals, and they broadly fall into the category of iterator API design. This is the thing I promised we would come back to. So the first proposal is proposal 61898, which is to iter package to golang.org/x/x for experimental. The idea is that this is going to have iterator adapters. For example, the classic map and filter.
Josh Bleecher Snyder:Maybe it'll also have concatenate. And the idea is, look, iterators now, we wanna be able to chain them and do things with them. This proposal has been stalled for quite some time now because the instant you start to use these iterator adapters, you discover a bunch of sharp edges in the language. Hey. I wanna use map with slices.
Josh Bleecher Snyder:But slices don't have built in iterators. Instead, you need to, use, slices dot values. Then you've done your map, and you wanna collect all the things at the end. Now you need slices dot collect. And so simply doing a map turns into slices dot collect of slices dot map of your slice and then some function where you have to write func blah blah blah blah blah and and then of slices dot values, and it turns out to be a lot less if you can just use a for loop.
Josh Bleecher Snyder:And so there's at least 2 issues here. One is that these things don't compose nicely, that is you have this sort of inside out nesting thing going on, and the other is they don't work with things that you can naturally iterate over, like all of the bytes in a slice or all of the elements, all the bytes in a string or all the elements in a slice. And so you might think, okay, we need a totally new, like, pipe operator to be able to compose these things the way that, Elixir has. Or you might think we need Lambdas so we can write these anonymous slice things very easily. And you might think, oh, and now we need adapters to get from, strings to byte.
Josh Bleecher Snyder:And by the time you've done, you've added, like, 3 or 4 major things to the language just to make an iterator adapter. And you've tried to write your own iterated iterator adapter package, what you've discovered, like me, is that you have dozens and dozens of these things that don't compose very nicely. And I don't have much to say about other than that it's a it's a large puzzle, and it's interesting reading to to poke through that proposal.
Jonathan Hall:What is the status of that proposal right now? It's it's still just an open proposal that's being discussed. Is that right?
Josh Bleecher Snyder:Yep. I think we are I can't say we. I think that the proposal is is open and hoping for more good ideas and more insights and to figure out, what is the minimal thing to do that is widely useful.
Jonathan Hall:Good.
Josh Bleecher Snyder:Alright. And the other interesting thing about iterators, which I think you might have some thoughts about, Jonathan, again, in the iterator API design space is proposal 70084, which is nominally about, having an iterator for os.file.readdir. That is, I wanna iterate over each of the files in a directory. Maybe, as happened in a previous company I worked at, you foolishly have millions of files in a single directory, and you wanna be able to iterate over them instead of grabbing them all in a slice. So now this is question.
Josh Bleecher Snyder:What should that iterator look like? And the whole proposal ends up being about API design for iterators with errors. It's really easy if you're iterating over the bytes in a string. You just get them 1 at a time. Nothing can go wrong.
Josh Bleecher Snyder:But what if as you are iterating through all the files in your directory, your reader call could fail. Should you have an iterator that is string comma error and then check the errors each way? Or maybe you should have a like result of type that contains both a string and an error, or maybe you should have an error checking function at the end like buffio dot scanner that you have to call at the end to see if any errors happened along the way. And I the current status, as I understand it, is nobody is happy with any of the options and knows how to deal with iterators with errors. So if you have experience dealing with iterators with errors or have written some code around that, go check out the the proposal and chime in with your insights because it's a giant puzzle.
Jonathan Hall:I I do have experience with this, and I don't know what the correct answer is. I have experience with this that predates the the new iterators in Go. And I think it's actually a broader problem than with just errors, and I'll give an example. In the, CouchDB library I maintain, there are certain iterators. So so basically, just to simplify what what the iterate does is it reads a string of arbitrary number of JSON blocks from the network, returning them 1 at a time.
Jonathan Hall:And then at the end, you use metadata. This could it might be pagination data. You're on page 6 of 1025 page items per page or whatever. Right? So it's not necessarily error data, but there can be some metadata that's only valid after the iterator is completed.
Jonathan Hall:And there's not any good way to solve that problem with our iterators either. You know, there's there's not a natural way to write an API that says, here's something that's available to you after you finish iterating.
Josh Bleecher Snyder:Yeah. So one of the proposals in, 70084 is to return an iterator and a func open close parens error. And so it is a function that you call later to gather your errors just the same way that you might return a cancel function from context that you call later to cancel that context. And the idea there is that you're like, you I promise that later when you're all done with this other stuff, if you call this function, it will have a useful error for you. The obvious failure mode, of course, is you don't call the function or you can call it at the wrong time or it's it's definitely a foot gun.
Jonathan Hall:Yeah. So that's essentially the way the old integrators work with like SQL where you have this rows this rows closer or something where you're forced to call rows that close and everybody forgets the install windows to do it for you.
Josh Bleecher Snyder:Exactly. Exactly. Or buff.io.scanner. And in the end, you have to call scan dot error, and you forget to do it. And, and hilarity ensues much later than you think.
Jonathan Hall:I don't know the answer, but it's a good question. Excellent.
Josh Bleecher Snyder:Hopefully, somewhat one of our listeners does and can go enlighten us.
Jonathan Hall:That would be great. So, Josh, I understand you have a few other things you'd like to talk about. We're gonna stick around after the break to to chat some more. We have a couple more items to talk about a little bit less. I was gonna say hard news, but this isn't hard news.
Josh Bleecher Snyder:It's this is this is just stuff I wanna share.
Jonathan Hall:Yeah. And we have a few lightning round things to discuss as well. So stick around till after the break for that. Thanks for listening to PepaGo. This show is a hobby.
Jonathan Hall:It's a little bit of an expensive hobby. We have to pay for hosting, editing, domain registrations coming up again, in a week.
Josh Bleecher Snyder:And you shouldn't believe the fee that I get for coming on as a cohost.
Jonathan Hall:Yes. Yes. We have to pay our cohosts a ton.
Josh Bleecher Snyder:0¢ $0, £0.0.0.
Jonathan Hall:That's right. That's right. So many zeros in there. We need to cover those somehow. And one way you could do that is becoming a by becoming a Patreon.
Jonathan Hall:You can actually also support us on Patreon for 0 if you want to, or you have the option to to send us $8 a a month on Patreon.
Josh Bleecher Snyder:Helps us
Jonathan Hall:a lot to cover these fees. You can also just share this show with a friend or a colleague or a fellow student or your mom or anybody else who might be interested in Go or cute gopher logos. We'd appreciate it. You can find our contact details at cupago.dev. That's the website where you can listen to all past episodes.
Jonathan Hall:You can see the transcripts of some episodes. I think we keep forgetting to add transcripts on all the new ones, but, some of them and one day, all of them, when we remember, you can find the link to our Patreon, to our Spive store where you can buy a cup of go cups. And you can also find us on Slack, on the go for Slack community in the cup of go kebab case channel. Join us there. And if you would do us the favor, leave us a rating and review wherever you listen to this podcast.
Jonathan Hall:We would appreciate it. Stick around. We're gonna talk a little bit more about you know, I don't even know what Josh. Josh has a surprise for me. So stick around.
Jonathan Hall:I'll be giving you surprises as you listen to it. Excellent. Alright, Josh. Hit hit me. What's the surprise?
Josh Bleecher Snyder:I have 2 things to share with you because it is the winter holiday break, and everyone is gonna be bored and wanting to tinker with things. The first, I heard you like podcasts. Sometimes. Yeah. We are lucky enough to, to work in the Go ecosystem, which can support, lots of sort of media and people like you doing the hard work of bringing the information to everybody.
Josh Bleecher Snyder:But there are lots of tiny projects that you might be interested in keeping up with that, don't have the sort of giant ecosystem around like that. So I have been helping, and I I say that loosely. It's not mostly me. I've been helping with a project called pi.dev, that is p i.dev that lets you take a GitHub repository and follow it like a podcast. It'll, do a roundup of all the issues and the pull requests and the work that's been going on and feed it to you in a, technically dense podcast.
Josh Bleecher Snyder:So if you wanna know what's happening with, say, the JSON v 2 package, you can sort of passively follow it.
Jonathan Hall:I listened to the the episode you shared with me, and I was quite impressed. I didn't actually realize it was just AI at first until it said, omid temti.
Josh Bleecher Snyder:Yeah. Pronunciation is hard. But we're we're adding in fact, the reason if you go to pie.dev right now on Thursday, you won't see anything is because we're tweaking a few little things like making it clear that it's AI. Yeah. That was that was an important thing for us.
Jonathan Hall:But by
Josh Bleecher Snyder:Friday, when people are listening, you should be able to go to pie.dev, drop in any GitHub repository you like, and and listen to it. So we'd love this is very much alpha quality. I wanted we've been rushing it to get it out so I could share it with you here. So, I'd love to hear feedback.
Jonathan Hall:Yeah. I have a link to the show in the show in the show as to. Excellent.
Josh Bleecher Snyder:And then the other toy, which, I heard you like Go.
Jonathan Hall:I do like them. Yeah. Sometimes. When I'm not iterating over things that might return errors.
Josh Bleecher Snyder:Exactly. Definitely lots of errors. This is also alpha quality software, also something that I have been helping with. And I still write code in the Go Playground sometimes because it's sort of like Zen mode. Right?
Josh Bleecher Snyder:Like, all of my project goes away, and I have a small thing that I'm hacking on, and I don't wanna think about it. The problem with Go Playground is that that the editor is kind of clunky, and I don't have all of the things that I want at my fingertips that I have in my other tools. I also like iterating on things, with Claude, with an AI assistant, but then I don't have a good editor, and I'm forced to do chat. And so we have cobbled together, and we'll be improving over time a tool at sketch.dev, like, don't draw the owl, sketch it. And it is a better glo go playground.
Josh Bleecher Snyder:So you've got a real Monaco editor and you've got a chat window. Jonathan's trying to go there, but it's not gonna work yet. It'll work on Friday. I promise. I swear what could go wrong?
Josh Bleecher Snyder:So you can go and you can work on Zen mode on a little piece of code, get it polished, get your tests working just the way you want, have some AI assistance, and then backboard it back into your project when you're ready.
Jonathan Hall:That's cool. I'm looking forward to trying it on Friday, which is today.
Josh Bleecher Snyder:Totally. Yeah.
Jonathan Hall:Through the magic of video of audio editing.
Josh Bleecher Snyder:Jonathan and I had had time zone snafus, and now we're having date time snafus. We we we better call it a day while we're still ahead or behind. How far ahead are we or behind?
Jonathan Hall:Yeah. So to the the baseballs. We're at now now, the scene again. I I wonder
Josh Bleecher Snyder:if I can I can taunt Filippo into into giving us Back to the Future music? What what else can
Jonathan Hall:we Yeah.
Josh Bleecher Snyder:Alright. So anyway, if you wanna deal with 1100 gigawatts or whatever it is, yeah, check check out sketch.dev, check out pie.dev. Give me feedback on the have we have we, talked about the Cup of Go Slack? That's a great place for feedback at the Cup of Go Slack.
Jonathan Hall:Well, while we're in the mood, in in the mode of talking about interesting projects, let's round out this lightning round. Lightning round. Not retrospective retroactively. I wanna talk about bifrist. So bifrist, which is not spelled the way you think.
Josh Bleecher Snyder:I mean, it is if you're Icelandic.
Jonathan Hall:If you're Icelandic, you you know exactly how to spell it. Yes. B I f r o with the little dots above it.
Josh Bleecher Snyder:That's a bummer, Jonathan.
Jonathan Hall:It is it? It might be. That's an that's an
Josh Bleecher Snyder:that's totally I mean, I know
Jonathan Hall:what it looks like. I just I never remember if it's actually like, those dots are always in or if it depends on context, whether it's
Josh Bleecher Snyder:anyway. So so Bifrost, what we as long as we talk about Iceland, there's a town called, Bifrost in Iceland. It's extremely boring. I've been there, but it's from the, the Norse mythology. It is the rainbow road that connects Midgard, where we all live, to Asgard, where the gods live, and where Valhalla is.
Josh Bleecher Snyder:And that's a great name for what it is. It is a highly custom customizable SSH server, so you can connect where you are as a as a peon to where the gods live in your servers. Or actually, I guess it provides the gods themselves because it is the SSH server.
Jonathan Hall:Yeah. Yeah. And it's written in Go, of course, or else we wouldn't be talking about it.
Josh Bleecher Snyder:Yes. And if you find yourself wanting an SSH server that is, less highly customizable but already built, but more like, hey. I just want my SSH server to do 1 or 2 things, and I want to design it from scratch, and you want, like, a nice net HTTP style interface. An older project, but still excellent, is glider labs slash SSH that provides the raw tooling for you to build your own SSH server in Go.
Jonathan Hall:Pretty cool. I could have used that, a few weeks ago. I was trying to, I was trying to build an SFTP client, and I wanted to write some tests. So I ended up wiring together a horrible, minimal SFTP server that I could run-in process for my tests. I bet that would have made it easier.
Josh Bleecher Snyder:I bet Glider Labs and Bifrost would both have helped.
Jonathan Hall:One last lightning round item I wanna pull up. This is something I just learned about today, although it's been around for at least a couple of years, and that is something called Go recipes. You may be aware of Awesome Go, which is a collection of Go projects. Go recipes is kind of the same thing. It's updated less frequently, apparently.
Jonathan Hall:This seems to be the 2024 updated version, so I guess it's updated maybe annually. But it's a list of links to all sorts of Go related projects, not necessarily Go code, but there's links to there are links to code. There's links to IDE extensions. There's links to all all sorts of things in here, cogeneration. Anyway, it's a long list of curated and organized by category tools and resources to help you with your Go projects.
Jonathan Hall:So I'll link to that in the show notes. Also, GoRecipes.
Josh Bleecher Snyder:I look forward to 2025 when awesome GoRecipes and GoRecipes for awesome come out.
Jonathan Hall:Can't wait. I guess that's what he'll be working on.
Josh Bleecher Snyder:No comment.
Jonathan Hall:All right. Well, thanks, Josh, for joining us on the show. I'm gonna get for Shai while he's gone. A reminder that we will not be recording next week. Enjoy your Christmas.
Jonathan Hall:Enjoy your Hanukkah. Enjoy your time off, whatever you're doing. We're we're working if that's you. I'm sorry if you're on call next week. We'll see you, I was gonna say, in the New Year.
Josh Bleecher Snyder:Enjoy your solstice. Enjoy the return of the sun if you're in the northern hemisphere and the waning of the sun in the southern. And, it's always a pleasure. Thanks for having me back, Jonathan.
Jonathan Hall:Thank you. Until next year.
Creators and Guests
