embeddings-splitter/lib/exampleLongPrompts.d.ts

A typescript library to split your long texts into smaller chunks to send them to OpenAI Embeddings API

export declare const example1 = "Prompt Engineering\nLearn how to use AI models with prompt engineering\nHow to get Codex to produce the code you want!\nHave you seen AI models that can generate code for you? Well, if you haven\u2019t, you\u2019re going to see them a lot more soon thanks to models like OpenAI\u2019s Codex models. Codex is a family of AI models from Open AI that translates between natural language and code in more than a dozen programming languages. The power of these AI models is that you can quickly develop and iterate on your ideas and build products that help people do more. Here is an example how you can have a conversation with a Minecraft character and have it follow your instructions by generating Minecraft API commands behind the scenes.\n\nMinecraft intro\n\nThis article will show you how to get models like Codex to generate code you want using a technique called Prompt Engineering. Prompt engineering is the practice of using prompts to get the output you want. A prompt is a sequence of text like a sentence or a block of code. The practice of using prompts to elicit output originates with people. Just as you can prompt people with things like a topic for writing an essay, amazingly you can use prompts to elicit an AI model to generate target output based on a task that you have in mind.\n\nPrompt Completion\n\nLike a person writing an essay, an AI model takes a prompt and continues writing based on the text in the prompt. The new text that the model outputs is called the completion. An example task might be to write a Python program to add two numbers. If you write out the task as a Python comment like so:\n\n# Write a function that adds two numbers and returns the result.\nAnd give it that comment as a prompt to Codex, it will generate the code as the completion for you like this:\n\ndef add(a, b):\n    return a + b\nThe easiest way to get started with OpenAI is to use the OpenAI Playground. This picture shows the Playground after Codex has generated the completion for the prompt in the comment.\n\nOpenAI Playground\n\nThe best way to learn how to use OpenAI models is to try them out. Check out the Getting Access section below to learn how you can get started. OpenAI has a number of examples for how to use their models including code examples. For Codex, the best practices documentation is a great resource for understanding how use to it.\n\nCodex also powers GitHub Copilot, an AI pair programmer available in Visual Studio and Visual Studio Code. Copilot draws context from code and comments you\u2019ve written, and then suggests new lines or whole functions. I wrote this article in VS Code, and as I wrote the above Python example, Copilot automatically used Codex to suggest the code. Below is a screenshot of Copilot in action. The grey text including the Python function is Copilot\u2019s suggested completion. Note that it even suggests the close quote for the Markdown code section.\n\nCopilot completion example\n\nSo how can you apply the power of models like Codex in your applications? An example like the one above is simple and easy for Codex to generate. For custom applications, you may need to craft the prompt to better describe your problem. This includes giving Codex examples to help tell it what you are looking for. The rest of this article shows you examples and techniques in prompt engineering to help you get the code you want.\n\nTell It: Guide the Model with a High Level Task Description\nYou saw above how you can tell Codex what you want and it will generate it. Codex is amazingly capable at generating code. The quality of its completions depends a lot on what you tell it to do.\n\nFor starters, it is usually a good idea to start your prompt with a high-level description of what you are asking Codex to do. As an example, let\u2019s say you want Codex to generate some Python code to plot data from a standard dataset. We could describe our task like this:\n\n# Load iris data from scikit-learn datasets and plot the training data.\nCodex is likely to respond with something like the following. Note that it generates code that assumes the scikit-learn datasets package is imported as datasets.\n\nRaw prompt\n\nTo fix this, you can tell Codex a bit more detail about what you want it to do. You can tell it that you want Python output and give it additional instructions as well, such as loading libraries first before using them.\n\n# Generate a Python program following user's instructions.  \n# Be helpful and import any needed libraries first.\nNow the output is more reasonable! It loads the expected libraries first before using them. Adding import into the prompt also helps Codex know that you want to import libraries first.\n\nCompletion with high level task description\n\nThis pattern combines a high level task description with a more specific user instruction. Together these are passed to the model as the prompt, which returns a completion.\n\nHigh evel task description and input\n\nShow It: Guide the Model with Examples\nPrompts are the input sequences given to an AI model, which generates the output word by word. For a given model, the exact sequence of text that you provide to the model in the prompt influences all of the subsequent output.\n\nSuppose that you prefer a slightly different style of Python code from what Codex generates. For example, to add two numbers, you like to name your arguments differently like this:\n\nFunction example\n\nThe general principle in working with models like Codex is that you tell it what you want it to do. A highly effective way of telling Codex what you want is to show it examples. Then it will work hard to match its output to the way you like. If you give Codex a longer prompt including the above example, then its completion names the arguments exactly as with the example you gave.\n\nBasic prompt engineering\n\nZero-shot, one-shot, and few-shot learning\nAs you work with large AI models, you may hear terms like zero-shot learning. Zero-shot learning is a prompt that has no examples - for example just a task description. One-shot has one example, and few-shot has more than one example.\n\nBelow is a diagram capturing the general pattern for few-shot learning:\n\nBasic with examples\n\nDescribe It: Guide the Model with High Level Contextual Information\nCodex is trained on a large amount of open source code. As you saw in the example with scikit-learn for Python, it knows about many libraries and packages in a variety of languages.\n\nWhat if you want to use a library that Codex doesn\u2019t know about? You know by now that the way to get Codex to do something is \u2026 to tell it or show it! Specfiically, you can describe the API library to Codex before you use it.\n\nThe Minecraft Codex sample uses Minecraft\u2019s Simulated Player API to control a character in the game. The Simulated Player API is a TypeScript library that lets you issue commands to move, chat, look at inventory, mine, and craft items. It\u2019s a newer API that Codex doesn\u2019t know about yet. Let\u2019s see how Codex does generating code for it for the following prompt:\n\n/* Minecraft bot commands using the Simulated Player API. When the comment is\nconversational, the bot will respond as a helpful Minecraft bot. Otherwise,\n it will do as asked.*/\n\n// Move forward a bit\nCodex tries to make an educated guess using \u2018bot\u2019 and \u2018Simulated Player\u2019 as cues:\n\nMinecraft code with raw Codex\n\nBut it\u2019s not correct Simulated Player code. We can show Codex what the Simulated Player API looks like using function signatures for the library. Below is a subset of the API definiition.\n\n// API REFERENCE:\n// moveRelative(leftRight: number, backwardForward: number, speed?: number): void - Orders the simulated player to walk in the given direction relative to the player's current rotation.\n// stopMoving(): void - Stops moving/walking/following if the simulated player is moving.\n// lookAtEntity(entity: Entity): void - Rotates the simulated player's head/body to look at the given entity.\n// jumpUp(): boolean - Causes the simulated player to jump.\n// chat(message: string): void - Sends a chat message from the simulated player.\n// listInventory(object: Block | SimulatedPlayer | Player): InventoryComponentContainer - returns a container enumerating all the items a player or treasure chest has\nWith the above API definition and some examples, you can get Codex to generate code that follows the API. Here are some examples using the API to control a Minecraft character.\n\n/* Include some example usage of the API */\n// Move left\nbot.moveRelative(1, 0, 1);\n// Stop!\nbot.stopMoving();\n// Move backwards for half a second\nbot.moveRelative(0, -1, 1);\nawait setTimeout(() => bot.stopMoving(), 500);\nNotice how the first example makes explicit what values to use for the moveRelative function signature. Once you have done that, we can ask Codex an instruction to move the character forward a bit. It will then generate the correct code.\n\nMinecraft completion\n\nThis pattern adds high level context in the form of the API definition, as well as examples to help Codex understand what you want it to do. It also allows you to use the API to generate code that is more correct.\n\nContext + Examples\n\nRemind It: Guide the Model with Conversational History\nFor real application use cases with natural language input, it is important to be able to carry the context of a conversation. Consider the following instructions:\n\nUser: I want to order a 12oz coffee\nUser: Oh and can you put it in a 20oz cup?\nThe AI model would need to know that \u201Cit\u201D refers to the 12oz coffee. This is easy for humans, but Codex is a fixed model: it doesn\u2019t change based on the user\u2019s input. Codex doesn\u2019t remember from API call to API call what the last prompt was or what it returned as the completion. How do we get Codex to know that \u201Cit\u201D refers to the 12oz coffee? That\u2019s right - we tell it what the user said before. What you can do is add the last input+completion pair as an additional example in the prompt. That gives Codex enough context to give better completions.\n\nConversational context\n\nLet\u2019s see this in action in another sample application: Codex Babylon. Codex Babylon is a Typescript application that uses BabylonJS to render a 3D scene. BabylonJS is a popular framework for 3D rendering. Codex Babylon provides a web interface to enter natural language commands to place and manipulate objects in a 3D scene. The commands are sent to Codex to generate code that places the objects in the scene. The code is then executed by the app in the browser.\n\nIn the animation below, you see the natural language input to make and manipulate cubes, the BabylonJS code that Codex generates, and then the output being rendered right in the browser. You can see that it correctly interprets the user\u2019s instruction to \u201Cmake them spin\u201D.\n\nBabylon Codex 2\n\nAs you might expect, it is not practical to keep adding to the input / output history indefinitely. The context has a limited size. So we keep a rolling window of the user input history in the prompt. This allows Codex to be able to deference things like \u2018it\u2019. We call this the session buffer. The Codex Babylon sample includes an implementation of a session buffer.\n\nPutting it all together\nNow we are ready to put all of this together. For a given application, you can provide Codex with a prompt consisting of the following:\n\nHigh level task description: Tell the model to use a helpful tone when outputting natural language\nHigh level context: Describe background information like API hints and database schema to help the model understand the task\nExamples: Show the model examples of what you want\nUser input: Remind the model what the user has said before\nThe boxes in the diagram below are to help you think about what goes into a prompt. The OpenAI models are very flexible and there are no rules that the prompt must follow a specific structure. You can intermix any of the parts (which may get you different results) and you can add or remove parts as you see fit. We hope to cover variations of these patterns in the future. Experiment and see what works for you!\n\nPrompt context\n\nBeyond the Basics\nI hope you have enjoyed this introduction to prompt engineering. Prompt engineering is a great way to get started and learn if Codex is a good fit for your application. You can use these same techniques with the other OpenAI models like GPT-3, as well as other foundation models or large language models. As you work to put your application toward production usage, there are a few things you may want to think about.\n\nHyperparameters\nYou can tune the behavior of OpenAI models with various hyperparameters such as number of tokens, temperature, and others. The hyperparameters are the parameters that define the behavior of the model. The hyperparameter temperature influences how the model completion may change each time, even with the same input. Setting the temperature to 0 should give you the same output each time, at least within the same session. The stop sequence is useful with Codex to stop it from generating variations of similar code. You can do that by setting the stop sequence to the comment sequence for the programming language you are working with. For example # for Python and // for JavaScript. Other hyperparameters are described in the OpenAI documentation.\n\nFine Tuning\nFine tuning is the process of using a dataset of curated prompt-completion pairs to customize a model\u2019s behavior for your use case. Fine tuning can increase the accuracy of the completions for your prompts. We won\u2019t cover fine tuning in detail here as it is not available yet for Codex. Here are links to general documentation for fine tuning on OpenAI and Azure.\n\nUser Experience\nUser preception can make or break the experience of what you build. Here are a few areas to keep in mind as you welcome users to your application.\n\nPerformance: Prompt engineering as we have discussed here can increase your prompt size, which in turn increases the latency of the completion responses from Codex. For production applications, you may need to decrease your prompt size to improve the perceived latency of your application. One way to do this is to fine tune your model. In addition to improving accuracy of the model completions, fine tuning has the added benefit that it may improve performance by making it possible to have good results with shorter context in the prompt.\nInteraction Design: How you guide users in their interaction with Codex can have a significant impact on their success with using the model. This is especially true because the output of the model is not always reliable. For example, Copilot makes it easy for users to reject Codex\u2019s suggestions because a user has to actively accept Codex\u2019s output. Because VS Code is an IDE, it\u2019s easy for users to edit Codex\u2019s output if it needs fixing.\nResponsible Use: Large models such as OpenAI are trained on internet data, and can reflect the biases in the training data. To use Codex or other OpenAI models in production, you need to do things like content filtering. Both OpenAI and Azure OpenAI Service offer capabilities to do this. The OpenAI content filter is described here.\nPrompt Engineering as \u201CSoftware 3.0\u201D\nWriting prompts is a new skill in how to build software. Andrej Karpathy, the head of AI at Tesla, shared this picture that designing prompts is \u201CSoftware 3.0\u201D in a Twitter exchange with Chris Olah.\n\nSoftware 3.0\n\nWith what you learned in this article, you can take your first steps to become a Software 3.0 wizard :D!\n\nGetting Access to Codex, Copilot, and OpenAI\nSigning up for OpenAI is easy. Codex is currently in limited preview: registered Microsot Build 2022 attendees can sign up to use Codex and other OpenAI models for free. See the details here. They can also sign up to use Copilot from the same page.\n\nMicrosoft also offers OpenAI models in the Azure OpenAI Service with Azure\u2019s enterprise capabilities of security, compliance, and global reach. Azure OpenAI Service follows Microsoft\u2019s principles for responsible AI use and offers tools to help you offer these models responsibliy to customers. You can learn more about Azure OpenAI Service and sign up here.\n\nNext Steps\nThere is no substitute for playing with the models yourself to learn how to use this exciting new technology. You can also look at these samples to learn about building Codex based applications in more detail, including prompt engineering.\n\nCodex for Minecraft Non-Player Characters\nCodex Babylon\nCodex Command Line Interface\nWe can\u2019t wait to see what you build with Codex to help people achieve new things or to do things more quickly! If you have any questions or comments, feel free to reach out.\n\nContributions\nKeiji Kanazawa (GitHub, Twitter) wrote this article. Ryan Volum helped me and many others understand prompt engineering and implemented many of these techniques in the sample applications. Dom Divakaruni, Seth Juarez, Brian Krabach, Jon Malsan, Jennifer Marsman, Adam Zukor and others reviewed this blog post and provided valuable feedback.\n\nPrompt Engineering maintained by microsoft\n\nPublished with GitHub Pages";
export declare const example2 = "let's talk about cupid thanks lovely introduction gentlemen um\ni will i will try to make this make sense\nso yes as um as as as uh dan was saying\nthis this whole idea started uh with some critique of solid and it was in a\nplace called pubconf which is a it's a kind of fun after party\nconference um from ndc so ndc.net developer conference is now much broader\nthan net but that's kind of its roots in oslo and it travels around the world\nand i was in i think i was in london and pubcomp is a kind of it's a little mini event that happens afterwards and\nthey take a load of the speakers and you have these five-minute lightning talks so the idea is you have it's called an ignite style talk so they\nhave 20 slides in um in five minutes and they run 15 seconds per slide and it just auto\nadvances okay and i thought well i'm going to talk about solid\nand the reason i want to talk about solid is i'm lazy and it has so solid is five um\nprinciples of software and i figured what i would do is i would spend one 15\nsecond slide describing the principle another 15 second slide critiquing it and a third one saying what i'd do\ninstead and that's 15 slides and then you top and tail and that's 20 slides so basically the the talk wrote itself\nso i figured i figured that would be fun and then as i was writing the talk i realized that my advice for each of\nthese five principles was the same which is don't overthink things and just try and write simple\ncode and keep things simple and and during the talk i describe what i what i mean by simple\nbut anyway so um and it went across well it was received well and then i made the crazy mistake of\nposting my slides online and lots of people got very cross because they had no context for the talk and they just\nsaw me on the internet saying something about solid which they cherish as a religious totem and so\nthey came for me with pitchforks just very briefly then solid is\nfive um principles of software that robert martin put together in the 90s\nhe actually has loads of these you've got well over 100 of these and the first five um it turns out you can arrange them to\nto spell the acronym solid so you have s single responsibility principle which is that code should have one\nreason to change o is open and closed principle which is about when when and how you should\nchange source code l is list substitutability\nprinciple which is about type systems but oo people think it's about objects and class hierarchies\ni is interface segregation which is when you have a massive bloated interface you\nshould turn it into smaller interfaces and d is dependency inversion which is\nhow you should um construct things and so for each of these i\ni kind of proposed a a a critique and some of them were more uh\npointed than others if you like more opinionated than others and in each case i was saying let's just\ndo something different so this was 2017 i think so then last year uh 2021\nbeginning of 2021 there was there's a meet-up called the extreme tuesday club\nand it's the i i think i'm right it's the oldest xp meet-up in the world\num and it started in 2001 or something and they've been meeting every week in a\npub in london during the pandemic of course they were meeting virtually and they still meet virtually\nor occasionally they have hybrid meetups and the xtc the extreme tuesday club\nuh they contacted me and said look we're doing a session on on solid it would be great if you came along\nso i said yeah sure i know there's some folks there that i know and it's a lovely group so i so i said okay i'll come along and\nand do that and just before the event one of the organizers phil nash he said um\nokay so you clearly you're not a fan of solid what would you replace it with and then there's kind of like the\nbroader question of are there universal principles of software are there universal things that you would\nreplace um that you could describe uh or or is it just all a bit nebulous\nand that doesn't work and so i did i came up with an acronym and it was it's actually a backroom\nso um so a backroom is where you start with the word and then you you make up\nthe things to fit and it was it was february it was near valentine's day\nand so i i decided i was going to use the word cupid because it has five letters and because it's like the\nopposite of you know solid macho cupid is like romance and and hearts and\nflowers and and i figured that the the software craftsman bros would really\nnot like a thing where cupid is the center of it so so now i had to come up with five things\nthat were that made the acronym cupid so i did this and then and it went\nacross really well i got some really good feedback and really encouraging feedback and then what happened is because most of my work\ni've discovered is based on other people's deadlines right so i i wouldn't have come up with cupid but\nsomeone challenged me to i wouldn't have then started talking about it except that i agreed to speak at a conference a\nfew weeks later and thought i should talk about this so cupid had a very\nuh humble uh yeah and i didn't sit there you know mulling on this thing for months and\nmonths and fairly simple beginnings but then what happened is it kind of picked up some momentum and\npeople started talking about it and i was going to multiple conferences at all virtual conferences during last year\nand speaking about cupid and and it got picked up and it was really it it's it's got some crazy momentum so\nabout a year later so this is now early 2022 um it appears on on on hacker news\nso it's down here on the front page of hacker news is my article about um about cupid and\nreally frustratingly it happens on the 15th of february so it's one day after valentine's day that it appeared on the\nfront page of hacker news because that would have just been perfect um but so suddenly people are taking\nnotice and then i discovered that cupid is now on the thoughtworks radar\nso the thoughts radar you've probably come across it's published every i think quarter\nand they talk about things that you should adopt which is they say this is a good idea\nyou should do this assess which is this is a good idea but it's new so you should at least take a look at it\nhold i think which is their their language for stop doing this because it doesn't work\nand cupid appeared as assess so we think it's a good idea and you should take a look which is really exciting\nand then i um i i looked at my website analytics and it turned out that\num so my website kind of flat lines like this then it just goes bang because i'm not very good at\npublishing content i do it very infrequently but it seems to be popular when i do so so i suddenly had in like a handful\nof days i had like 50 000 um hits on the website and uh\nyou know as you can see uh this is what happens when you appear on the y combinator on\nhacker news is that 10 000 people hit your website um twitter was kind of there's a lot of\ntwitter activity and then thoughtworks these are all the click-throughs from the thoughtworks radar\nand so it was quite fun um so i discovered a that my static website\ni'm using hugo now i've moved away from wordpress and my static website hosted on github pages can handle spikes yes\nand also that my um my analytics software um fathom\nis is very good at handling spiky traffic and telling me about it so um so there we go so this is like a\nyear i guess a year and a bit now after i started talking about it and and it seems to be picking up\nmomentum this is really exciting because i love talking about good software so\nwhat do i mean by good software let's set the scene let's set the scene by asking who we are\nwriting code for now this is one of my absolute favorite quotes and i came across this probably\nlate 90s um in martin fowler's book uh refactoring\nuh which he published in 96 uh he has he in the opening preface maybe or certainly in the\nopening chapters he says this he says any fool can write code that a computer can understand\ngood programmers write code that humans could understand and i had this\nepiphany it was wonderful i had this moment where i thought wow you know i'm not hacking words into a\ncomputer to make it do stuff i'm writing code for other people\nto work with and other people might be future me right so this is about self-care as well\nand so in the context of this i'm looking at this i thought understand is an interesting word but it's a pretty\nlow bar you know someone can understand this code if they stare at it for long enough can we do\nbetter than understand and then a few years ago uh i was at a\nkevin henney talk and kevin henney does brilliant talks where he really researches things you know and he\nso he has lots of references and he started talking about habitability of code and he said habitability um\nand he uses dick gabriel quote so richard gabriel and one of the great software thinkers\nhe wrote a series of of essays and he published them in a book called\npatterns of software in 1996 96 apparently was a good year for software writing\nso patterns of software series of essays please please go and read this they're all in the public domain they're all\nhosted on his website and this one is called habitability and\npiecemeal growth and it's just a beautiful piece of writing so do yourself a favor go and read habitability of impeachment growth\nand just take a moment to appreciate that there are people out there who really care about how human beings\ninteract with software and he says this habit of habitability is the characteristic of source code\nthat enables and then he says a whole bunch of different roles enables people to understand its construction\nand intentions and to change it comfortably and confidently i thought oh comfortably and confidently\nwhen's the last time i felt comfortable and confident in someone else's code he says habitability makes a place\nlivable like home and i thought that's great so habitable sounds better\nand i thought what habitable again is kind of it's it's you can get by day to day what about joyful\nright is there have you experienced joyful code so i just want you to take a moment now\nand ask yourself have you worked on a code base that is joyful to\nwork with that just you you crack open the code and you smile you look at it and you say\ni'm going to be okay here this is going to be today is going to be a good day right you're going especially\ninto an unfamiliar code base you've got trepidation you're nervous you're oh no this is going to be such a nightmare and\npop it open and oh wow this all makes sense this all just fits in my head this i get this\nso what makes it joyful can we describe what makes it joyful\nand then in particular what kinds of properties does joyful code have i'm being very\ndeliberate with my language here so solid is based on principles it's a\nset of five principles out of a as i said a collection of well over a hundred\nsoftware development principles design principles um\ni i found that as i was thinking about principles they didn't really work for me so i found that i ended up looking at\nproperties and the reason for that is this is that principles are kind of rules\nright so they define the condition the conditions or the boundaries for something\nso they say what's inside and outside so your code conforms to to the conditions or it is wrong and you see this a lot or\ni see this a lot with particularly young male programmers\nfresh out of college coming into their first or second job and they look at a code base and they go\noh this is all wrong and say what what's wrong with that oh this violates the single responsibility principle this\nviolates the principle of least surprise this violates the digital and and so all you know they're going\naround policing the code and you're just thinking honestly there's must be better things to do with your time\nso yeah your code conforms or it's wrong right and those are your options\nso properties are more like qualities or characteristics so they define a center\nso the idea is that your any code is like nearer to or further from that center\nbut it's never like wrong right so you could say well based on these three if we think about what this characteristic\nof this code this code expresses this characteristic a lot or not very much and it gives us a sense of direction as\nwell it says if i want to change this code to exhibit more of this property to be closer to the center what do i need\nto do so i'm going to get a bit meta from the moment\ni want to talk about the properties that properties should have or at least when i was formulating cupid\nwhat did i want how did i want to describe each of these properties and i came up with three things that i\nfigured if i were hearing these things for the first time how i would want someone to describe them to me\nso the first thing is i want them to be practical right i don't want abstract things i don't want deep philosophy i don't want\nsemiotics i don't want any crazy i want things that are easy to articulate so they're easy for someone to describe to\nme and they're easy for me to describe to you easy to assess so i can look at some\ncode and say does it exhibit to what extent does it exhibit this property\nhow close is it to the center and also easy to adopt i didn't want like a\nhard and fast set of rules that are all or nothing i want to be able to look at some code\nlook at these properties and say right let's start shifting this piece of code in this direction and that is\na piecemeal easy local set of changes\nso i don't need to convince everyone to come on this big adventurous journey with me i can just make small changes to\nmake the code more habitable more joyful the second thing is i want them\nhuman what i mean by that is and if and so that again solid is a\ngreat example of this solid it is principles written from the perspective of code it's what code should be and what code\nshould have whereas these properties are from the perspective of the human beings working\nwith the code so when you look at code the extent to which it makes you joyful\nis a subjective experience right so so the properties that i want to describe are human they're about you and\nabout your interaction with code and the third thing is i i like the idea\nthat they're layered so they're nuanced so when you're first describing some of these properties to\npeople they're going to go okay that that seems familiar that seems to make sense but\nthen as you get more experience as you get deeper into software as you look at more and more code bases you work on\nmore code you gain more experience you can revisit these properties and see more and more texture to them more and\nmore nuance to them so that said here's cupid so i'm going to\ngive you my five properties my five characteristics of joyful code and then i'm going to spend\na bit of time unpacking each one okay so composable i like code that's\ncomposable that plays well with other code i like code that follows the unix philosophy of doing one thing well\nso it's comprehensive and it's single it's single purpose it's focused\ni like code that does what i think it's going to do i like code that's predictable and i'm describing predictable as a\ngeneralization of testable so predictable is code that does what you think it does\nidiomatic idiomatic code looks like code everyone else writes idiomatic code is\nlist code so i'm not going to write code and have people go ah that must have been written by daniel because that's daniel's code\nstyle i want to write code where people look at it and say that was written by someone who is an\nexperienced kotlin programmer or an experienced c-sharp programmer or an experienced uh rust programmer\nthat they've they understand the idioms of this language and they are writing idiomatic code they're not trying to\nwrite and you'll see this in dot net all the time when people move when people first start using f sharp for instance from c\nsharp they will write c sharp in f sharp they'll write they'll try to write you\nknow fun um method type things and object type things and class type things in f sharp\nwhile they're getting the hang of thinking functionally they'll want variables because they don't they're not\nused yet to the idea of immutable values and transformations\nthey will want to iterate over things where you might have a map reduce or a\nrecursive function or a closure because those concepts are new so you tend to find i did this when i\nwas learning python my background was in java and i was i had java class or java style classes and objects all over the\nplace and it turns out the idiomatic python is much much more lightweight than that and\nwhat i found over time is i was just deleting most of the code i'd written and just leaving what i needed and i ended up with much much less code and it\nlooked like python code now instead of looking like someone trying to write java in python and finally domain based\nso this is code that uses both the language of the domain and is structured for the problem i'm solving\nso let's take a look at each of these in turn then uh composable\nso code that's composable it code that's easy to use gets used and\nused and used okay so i want code that's easy to pick up easy\nto use what makes code composable having a small surface area makes code composable\nso if i look at n unit or j unit or any of those kind of\nunit testing things in order to write a test um i need a test case and a fixture and\na test class and the class has methods on it and\nthe methods have to have an attribute on them to say that it's a test method and so on right so there's a bunch of\nstuff i need to do to use to use this and that there's a there's a learning curve ahead\nassociated with that in python there's a lovely little test framework called pi test\nand literally the way you write a test is you write a function whose name starts test underscore\nand it will run as a test you don't even have a certain you don't have a cert equals\nright um because the word assert the keyword assert is baked into the language and so what pi test does\nis you just use the keyword assert with like just double equals or doesn't equal or whatever your comparison is and if\nthat assert fails it prints you out this very very readable error message it says this is a\nline of code these were the two values this is why it didn't work and also this is where these values were assigned so\nyou can go straight into the code to where the problem probably is and once you've used something as\nlightweight and powerful as pie test you look back at the things you've been using all the time and say\nwow you know i've gotten so used to all this other stuff that i had to drag in\nsmall surface area less for me to learn less for me to get wrong\num and also less to conflict one of my very first um uh\nopen source projects back in 2003 i reckon it was\nwas um some code that uh it tests a really horrible enterprise java thing\ncalled enterprise java bean if you've never heard of enterprise java beans you are living your best life okay\nso this code tested enterprise java means particularly for a thing called ibm websphere which again if you've\nnever come across it thank me um but the um\nso the the idea that there was um less to conflict so i had some logging in there\nokay and the problem with the logging is i was using a logging library and that means that if you wanted to use my thing\nyou had to use the same logging library at the same version and so now that was a barrier to entry and so\na my colleague pointed out that i had this dependency and i was like so what is just logging but then i remove that dependency and\nsuddenly people started using this um as open source code because they could now\nand the reason they hadn't been was because they got stuck so composable code is has an intention\nrevealing name i can find it right there's no point you writing a brilliant brilliant lovely little\nsmall surface area library for me to use in c sharp and putting it out there on nougat\nor and and um and i can't find it because you gave it a really clever fancy name instead of\njust describing what it does and there's a lot of fancy names out there so give me an intention revealing name\nmake it easy for me to find and make it easy for me to evaluate let me make let\nit be very easy for me to determine if this isn't what i want if it's if i can find out within two minutes that this is\nnot what i want you just saved me a load of time and i will thank you and again what minimal dependencies so\nwe don't want it to drag in everything there's a lovely lovely web framework um called htmx and htmx is saying we've\nmassively over engineered things and over complicated things with javascript and all these heavy heavy javascript frameworks and\nyour reacts and reduxes and views and embers and\nuh angulars and all of these things right these big heavy frameworks just use html just use html it made a\ntiny extension to html so that you can have basically you can add behavior to let to\nany uh html tag rather than just two forms and buttons anchors\num really minimal the entire library is like 12k or something it's really tiny and it has\nzero dependencies so guess what it's really easy for you to pick up and use there's this wonderful quote from the\nlate joe armstrong who designed the erlang language and he's talking about oh here object-oriented languages here\nhe said you wanted a banana but you've got a whole gorilla you know i want to use some tiny little bit of behavior in\none of the methods on your class and so in order to get that i need to pull in your object its class definition\nall the superclass definitions all the interface definitions there's this ton of stuff that i get just so i can get\nthis one bit of behavior he said and that makes it really hard well that makes the decision to use your\nstuff much harder so there's composable we want small surface area intention revealing code\nand we want minimal dependencies so unix philosophy then unix has been\naround about as long as me right we're both in our early 50s unix started in\nat a university at uh no sorry i started in a telco\nin this in the late 60s and then it was adopted by university and then it went back out to the telco\nand it's been around a bit and then the the eunuchs you will most likely have come across started as a hobby\nproject by a finnish programmer who wanted to build a his own running\nkernel on uh on windows on ms-dos sorry at the time\nand that became linux and so unix has been around a long time and part of its longevity\nis it has this very very simple design philosophy so in unix everything is a file\nand files contain text or not text that's it right so when i say everything is a file\na file is a file a directory is a file a tape system if you're putting stuff onto\nonto tapes back in the day as a file hard drive's a file network device is a file the memory is a file\nanything so what that means is i can use file type semantics basically hierarchies of stuff and at the leaf\nnodes they contain things to describe anything on a running unix\nsystem and they had this philosophy they say make each program do one thing\nwell and there's two parts to that there's one thing and there's well so doing one thing means it's got a\nsingle purpose and doing it well means it's comprehensive i don't need to look anywhere else\nand that along with composability is is pretty powerful so\nlet's take a look at the ls command the ls command um lists file details that's what it does\nit's like i think it's a get object um in in powershell\nls list files content but here's the fun thing ls doesn't know anything about\nfiles okay all it knows about is the output of a command called stat and stat is the\nthing that knows about files so stats goes and looks at files or directories or whatever ls tells it to look at\nand it comes back with a bunch of structured records that tell you what those files contain\nand ls's command is to format those structured records into something on the screen\nso you see how these are very separate things stat just returns these structured records but they're really hard tons and tons of stuff in there and\nthey're hard for humans to interpret ls takes that and says i'm going to give you something that you can read as a human being\nso at ls if you look at the manual page for ns ls has a lot of parameters right a lot\nof switches because there's lots of ways you might want to look up details about files\nand so what that means is you don't need to go anywhere else to learn about um file content to learn about your\nfile listings you just use the ls command and yeah so together with composability\nthen there's really nothing you can't do so the idea that files are text and then\neverything is text is you can take this then to composability to say the output of any command should be text which i\ncan use as the input to any other command so unix gave us the pipe operator\nwhich is how we sequence things together so for instance i cat gives me the contents of a file and i can pipe that\ninto grep which is a filter so that filters things i can pipe that into said which transforms patterns it finds a\npattern and replaces it with something else i can then sort the results so now i get alphabetical output\nand then of that alphabetical output i can use unique which removes any duplicate rows\nand so on and so on and again powershell sort of adopted this idea\nbut it i i think it got it annoyingly badly wrong in that it tried to use well\ni did it introduced an object-oriented language into the command line instead of a\npipe oriented language so something so basically you can think of all of these as functions these are all transformational functions\nand so if you had a functional type of thing in powershell i think it would be a lot less difficult on the brain so we\nhave you know objects with methods and all this kind of stuff and there's this huge huge\ncomplexity to powershell that you simply don't have to a regular unix shell that has commands and the commands by\nand large use text and they manipulate text and pass text onto the next thing\nso but surely doing one thing well is the same as single responsibility principle i hear you say\nnot really so srp is about how the code changes the canonical description of srp is\ncode should have one reason to change and in my original talk i was saying that that's completely insane because\nany code could have many reasons to change it could have functional reasons it could have legal reasons it can have security reasons it can have operational\nreasons it can have many many reasons why it might need to change you know for any single line of code so\nthat's a bit of a non statement to me anyway but when you unpack it what they're really\nsaying which kind of makes sense at the time again solid is not a bad idea it's\na bad idea now in the 90s it was it was what we thought about object-oriented programming so it\nwas good advice for 30 years ago okay and 30 years ago you would often have\ndifferent people often completely different teams working on the same code and so\nwhat single responsibility said was please please don't have people from different teams hitting the same source\nfiles that's crazy right so break it out so that there's only one type of person who's going to\ngo in there and change this code so if i have a form on a screen then the ui\npart of that form uh the the the client part of that form is probably one team\nthe data access part of that form is another team over here with the dbas and data access and whatever else and then\nthere's a bunch of business logic probably involved in this form that a some business team would\nwould work on so srp makes sense in that context as soon as you have the same team all responsible for all of that and\noften the same person it's completely insane to say code should have one reason to change code\nshould have be changed by one person or the same people who are going to understand all of that great but you can\ndo all of that in one source file which will come to you later so srp is about how the code changes\nabout people of people changing the code not what the code does so it's an inside out perspective not an outside in\nperspective unix philosophy says don't care don't care how it got there it does one job and it does that one job\ncomprehensively and i thought i don't look anywhere else okay so\np predictability let's look at predictability of code one of my\nfavorite uh descriptions of predictability is kembek in amongst his his four rules of of good\nprogramming um is that it passes all tests right it works yeah\nso behaves as expected with no surprises it passes all tests i'm kind of broadening that definition i'm\ngeneralizing that definition to say even when there are no tests okay so it behaves as expected\nwhat i mean by that let me give an example of this um and this is one of the first code bases\ni ever worked on this is back in probably 91 i think and it was uh image processing software\nso digital image processing which is you take in a huge at that time huge digital\nimages and the images are made up of uh pixels that have four color\ncomponents so basically four four digits four numbers and the numbers stand for printing ink\ncolor so you have cyan magenta yellow and black cmyk and simon yellow and black are the\ncolors that you use when you are printing uh printed color printed uh newspapers\nmagazines using massive print rollers so i was working on software that made it\neasy to test the image that you were going to print before you printed it because once\nyou're printing it is very expensive to find out you've got all the colors wrong so what we had was basically a 4d color\ncube so it was a mapping of these sets of four colors you put this into this cube and it would come back out with the\nfour colors transformed and you did that for every single pixel in the exam in the image and then you\nnow have the color corrected image somewhere deep in this morass of very complex image manipulation code\nor image color manipulation code was a bug and that bug was assigned to me\ndan fresh out of college go and fit to the heart of this really big complicated system written in\nprobably hundreds of thousands of lines of sea maybe millions of lines of sea it was a big program and it's all written\nin c and find the bug and i was i'm not kidding i was\nterrified right so i crack open the code base i'm like where are we and i had that joyful moment\ni looked at the code and i was thinking this makes sense i get this i can navigate this\nand there was a brilliant tool open source tool around at that time called c tags and c tags was a way of basically\nbeing able to click through yeah it's kind of like um when you do control click now on on source code and you can go to references\nand go to definitions and find usages and all of that this was in vi in the early 90s and it was pretty\ngroundbreaking stuff so i could navigate the code through my editor\nand very quickly i got to where i thought the bug probably was no tests right just\nin very very simple well laid out well\nstructured code that i could navigate so i find out where i think this bug is and it really was one of these logic errors like a\nless than instead of a greater than or a missing equals or something like that and so i then fixed the bug\nand i was pretty confident that i fixed the bug i was pretty confident that i hadn't broken anything else i then rebuilt the code\nand took again no tests right no automated test we didn't do that then and i took a big image and i\nput it through the the now fixed code and it came out looking exactly like the original image\nbut the bug wasn't there anymore and i was like yes and i can't take credit for that right\nthat wasn't me that was a guy i'll tell you his name his name's phil davis and i learned a lot about programming from\nphil davis in the early 90s i don't know where he is now i'm sure he's somewhere on linkedin\nbut he he had this mindset of i'm going to write code that other people are going to want to work in\nand so it doesn't need tests to behave as expected\nokay and conversely some of the worst code i've seen was completely test