)]}'
{"/PATCHSET_LEVEL":[{"author":{"_account_id":1000066,"name":"Adam Joseph","display_name":"amjoseph","email":"adam@westernsemico.com","username":"amjoseph"},"change_message_id":"b74e61736b9934dd0a3605daf04c2bb97a630a9f","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":9,"id":"aabd907a_2827f031","updated":"2023-03-08 11:58:34.000000000","message":"I like this a lot.\n\nI have not read cl/8104 yet, but if the implementation lives up to the advertisement this is definitely the way to go.\n\nI have time tomorrow and intend to start reading cl/8104, but first I have to catch up on ~2 months worth of commits to tvix, so we\u0027ll see if I actually get to the CL.  I may upgrade this to a +2 after I\u0027ve seen the gory details.","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000066,"name":"Adam Joseph","display_name":"amjoseph","email":"adam@westernsemico.com","username":"amjoseph"},"change_message_id":"ce949d80d23368ad8ecc5651a287e330d43c9421","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":10,"id":"b3d0e762_caa99835","updated":"2023-03-11 11:09:11.000000000","message":"As part of my review process for cl/8104 I\u0027ve been doing some search-and-replace naming, mainly to pick names for things that I find easier to remember as I\u0027m working my way through it.  Mainly not meant to be merged, but I\u0027ll push them as WIP CLs in case you find one or two worth cherry-picking.  Currently the chain runs up through cl/8255 but I\u0027ll be extending it further.","commit_id":"a0b4941cd559fea9a5f3a2042394443775d984e1"}],"tvix/eval/docs/vm-loop.md":[{"author":{"_account_id":1000066,"name":"Adam Joseph","display_name":"amjoseph","email":"adam@westernsemico.com","username":"amjoseph"},"change_message_id":"b74e61736b9934dd0a3605daf04c2bb97a630a9f","unresolved":true,"context_lines":[{"line_number":47,"context_line":"   previously need to recurse."},{"line_number":48,"context_line":""},{"line_number":49,"context_line":"We do this by making use of the `async` *keyword* in Rust, but notably"},{"line_number":50,"context_line":"*without* introducing asynchronous I/O in tvix-eval (the complexity of which"},{"line_number":51,"context_line":"is undesirable for us)."},{"line_number":52,"context_line":""},{"line_number":53,"context_line":"Specifically, when writing a Rust function that uses the `async` keyword, such"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"2fc2661a_aebffef8","line":50,"updated":"2023-03-08 11:58:34.000000000","message":"It is not a goal to introduce async I/O in tvix-eval.  I think we both agree about this.\n\nIs it a goal to introduce *concurrency*?  I\u0027m not sure.  But that is the big question here.\n\nNote that in order to have async I/O, you must have concurrency.  But the opposite is not true.  A toy example of concurrency without I/O would be interleaving the execution of subexpressions (for example `[ (f x) (g x) ]` interleaving the bytecodes for `f` and `g`) with context switches possible only where the frame type changes (i.e. when bytecode invokes a builtin or a builtin invokes bytecode).  This is a silly toy example just to make clear what concurrency-without-I/O means; it is not a motivation.\n\nI bring this up because the async I/O state machine is quite complex and tricky specifically because it needs to support concurrency.  Also, Rust\u0027s optimizer -- as awesome as it is -- will not be able to figure out that our `async`-keyword-using programs are not concurrent.  So we will lose a large amount of optimization opportunities as a result of this -- indirect calls where the jump target is always the same, etc.  See next comment about this.\n\nThe previous paragraph is really just my verbose way of venting about the fact that Rust doesn\u0027t have an explicit tailcall mechanism.  If it did, none of this would be necessary.  But it doesn\u0027t, and it won\u0027t get one anytime soon.  And the trampolines were an unreadable mess; I\u0027m happy to see them go.\n\nAnyways, now that my rant is over; to resolve this comment, would you please indicate if concurrency is a conscious goal here?  I suggest doing so by editing to one of the following, whichever is appropriate:\n\n1. \"*without* introducing asynchronous I/O or concurency\" or \n2. \"and making tvix-eval concurrent, but notably *without*...\"","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000001,"name":"tazjin","email":"tazjin@tvl.su","username":"tazjin"},"change_message_id":"46c62dd5b97259d624d775d71de7976fc2548064","unresolved":false,"context_lines":[{"line_number":47,"context_line":"   previously need to recurse."},{"line_number":48,"context_line":""},{"line_number":49,"context_line":"We do this by making use of the `async` *keyword* in Rust, but notably"},{"line_number":50,"context_line":"*without* introducing asynchronous I/O in tvix-eval (the complexity of which"},{"line_number":51,"context_line":"is undesirable for us)."},{"line_number":52,"context_line":""},{"line_number":53,"context_line":"Specifically, when writing a Rust function that uses the `async` keyword, such"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"d9d7e7ca_f47ebaae","line":50,"in_reply_to":"2fc2661a_aebffef8","updated":"2023-03-08 12:44:36.000000000","message":"Concurrency is not *currently* a goal, and I\u0027m not actually sure that it will be. We do aspire to be able to suspend and *go do something else* (while waiting on the store), but I think that\u0027s doable without actually doing anything concurrently, and probably even with the machinery already built for this.\n\nI\u0027ll update it to the first proposed alternative.","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000066,"name":"Adam Joseph","display_name":"amjoseph","email":"adam@westernsemico.com","username":"amjoseph"},"change_message_id":"b74e61736b9934dd0a3605daf04c2bb97a630a9f","unresolved":true,"context_lines":[{"line_number":54,"context_line":"as:"},{"line_number":55,"context_line":""},{"line_number":56,"context_line":"```rust"},{"line_number":57,"context_line":"fn some_builtin(input: Value) -\u003e Result\u003cValue, ErrorKind\u003e {"},{"line_number":58,"context_line":"  let mut out \u003d NixList::new();"},{"line_number":59,"context_line":""},{"line_number":60,"context_line":"  for element in input.to_list()? {"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"d8689d0a_ed943796","line":57,"updated":"2023-03-08 11:58:34.000000000","message":"I think you mean `async fn some_builtin(...`.\n\nIt\u0027s a bit confusing that the previous paragraph says \"a Rust function that uses the `async` keyword, such as \u003cfunction that doesn\u0027t use the async keyword\u003e\" 😊","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000001,"name":"tazjin","email":"tazjin@tvl.su","username":"tazjin"},"change_message_id":"46c62dd5b97259d624d775d71de7976fc2548064","unresolved":false,"context_lines":[{"line_number":54,"context_line":"as:"},{"line_number":55,"context_line":""},{"line_number":56,"context_line":"```rust"},{"line_number":57,"context_line":"fn some_builtin(input: Value) -\u003e Result\u003cValue, ErrorKind\u003e {"},{"line_number":58,"context_line":"  let mut out \u003d NixList::new();"},{"line_number":59,"context_line":""},{"line_number":60,"context_line":"  for element in input.to_list()? {"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"f92719c1_bc3a459b","line":57,"in_reply_to":"d8689d0a_ed943796","updated":"2023-03-08 12:44:36.000000000","message":"whoops, fixed :)","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000066,"name":"Adam Joseph","display_name":"amjoseph","email":"adam@westernsemico.com","username":"amjoseph"},"change_message_id":"b74e61736b9934dd0a3605daf04c2bb97a630a9f","unresolved":false,"context_lines":[{"line_number":81,"context_line":""},{"line_number":82,"context_line":"All \"communication\" between frames happens solely through values left on the"},{"line_number":83,"context_line":"stack: Whenever a frame of either type runs to completion, it is expected to"},{"line_number":84,"context_line":"leave a *single* value on the stack. It follows that the whole VM, upon"},{"line_number":85,"context_line":"completion of the last (or initial, depending on your perspective) frame"},{"line_number":86,"context_line":"yields its result as the return value."},{"line_number":87,"context_line":""}],"source_content_type":"text/x-markdown","patch_set":9,"id":"6bc4379b_6ca3bccf","line":84,"updated":"2023-03-08 11:58:34.000000000","message":"Hooray.  Let\u0027s document this convention in the code as well.  Trying to reverse engineer the stack discipline was the most unpleasant part of the half-working trampoline CL.","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000066,"name":"Adam Joseph","display_name":"amjoseph","email":"adam@westernsemico.com","username":"amjoseph"},"change_message_id":"b74e61736b9934dd0a3605daf04c2bb97a630a9f","unresolved":true,"context_lines":[{"line_number":107,"context_line":"    ┏━━━━━━━━━━━━┷━━━━━┓                ╭───────────┴───────────╮"},{"line_number":108,"context_line":"───►┃ frame_stack.pop()┃                ▼                       ▼"},{"line_number":109,"context_line":"    ┗━━━━━━━━━━━━━━━━━━┛         ┏━━━━━━━━━━━━┓        ┏━━━━━━━━━━━━━━━━━┓"},{"line_number":110,"context_line":"                 ▲               ┃ call frame ┃        ┃ generator frame ┃"},{"line_number":111,"context_line":"                 │               ┗━━━━━━┯━━━━━┛        ┗━━━━━━━━┯━━━━━━━━┛"},{"line_number":112,"context_line":"                 │[yes, cont.]          │                       │"},{"line_number":113,"context_line":"                 │                      ▼                       ▼"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"b1f8ac09_1499b1dd","line":110,"updated":"2023-03-08 11:58:34.000000000","message":"\"Call frame\" is a bit ambiguous; both types call something (either bytecode or Rust code).  Would you consider \"bytecode frame\" instead?","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000001,"name":"tazjin","email":"tazjin@tvl.su","username":"tazjin"},"change_message_id":"46c62dd5b97259d624d775d71de7976fc2548064","unresolved":false,"context_lines":[{"line_number":107,"context_line":"    ┏━━━━━━━━━━━━┷━━━━━┓                ╭───────────┴───────────╮"},{"line_number":108,"context_line":"───►┃ frame_stack.pop()┃                ▼                       ▼"},{"line_number":109,"context_line":"    ┗━━━━━━━━━━━━━━━━━━┛         ┏━━━━━━━━━━━━┓        ┏━━━━━━━━━━━━━━━━━┓"},{"line_number":110,"context_line":"                 ▲               ┃ call frame ┃        ┃ generator frame ┃"},{"line_number":111,"context_line":"                 │               ┗━━━━━━┯━━━━━┛        ┗━━━━━━━━┯━━━━━━━━┛"},{"line_number":112,"context_line":"                 │[yes, cont.]          │                       │"},{"line_number":113,"context_line":"                 │                      ▼                       ▼"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"72ddd245_11d49f11","line":110,"in_reply_to":"b1f8ac09_1499b1dd","updated":"2023-03-08 12:44:36.000000000","message":"I would, but I\u0027d also prefer to rename that after the chain around cl/8104 is done. I\u0027ve filed b/258 to track this so we can do it afterwards as part of cleaning up.","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000066,"name":"Adam Joseph","display_name":"amjoseph","email":"adam@westernsemico.com","username":"amjoseph"},"change_message_id":"b74e61736b9934dd0a3605daf04c2bb97a630a9f","unresolved":true,"context_lines":[{"line_number":123,"context_line":"                 │                         ┃ frame completed ┃"},{"line_number":124,"context_line":"                 ╰─────────────────────────┨  or suspended   ┃"},{"line_number":125,"context_line":"                                           ┗━━━━━━━━━━━━━━━━━┛"},{"line_number":126,"context_line":"```"},{"line_number":127,"context_line":""},{"line_number":128,"context_line":"Initially, the VM always pops a frame from the frame stack and then inspects"},{"line_number":129,"context_line":"the type of frame it found. As a consequence the next frame to execute is"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"799fbeec_ae4e977a","line":126,"updated":"2023-03-08 11:58:34.000000000","message":"Have you found a nice tool for drawing diagrams like this?","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000001,"name":"tazjin","email":"tazjin@tvl.su","username":"tazjin"},"change_message_id":"46c62dd5b97259d624d775d71de7976fc2548064","unresolved":false,"context_lines":[{"line_number":123,"context_line":"                 │                         ┃ frame completed ┃"},{"line_number":124,"context_line":"                 ╰─────────────────────────┨  or suspended   ┃"},{"line_number":125,"context_line":"                                           ┗━━━━━━━━━━━━━━━━━┛"},{"line_number":126,"context_line":"```"},{"line_number":127,"context_line":""},{"line_number":128,"context_line":"Initially, the VM always pops a frame from the frame stack and then inspects"},{"line_number":129,"context_line":"the type of frame it found. As a consequence the next frame to execute is"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"1b098f83_0f9c48d9","line":126,"in_reply_to":"799fbeec_ae4e977a","updated":"2023-03-08 12:44:36.000000000","message":"Well, Emacs, but it doesn\u0027t really help that much. I use artist-mode to get approximate initial spacing of the text, and then manually edit the boxes. I\u0027ve done this a lot so I\u0027m quite fast at it, using reference pages like http://xahlee.info/comp/unicode_drawing_shapes.html\n\nAs an aside, it\u0027s incredibly infuriating that none of the arrows present in Unicode match up correctly with the box drawing characters, see this horrible example: https://gist.github.com/tazjin/aea039475e2da8a890955ec20c91f22a","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000066,"name":"Adam Joseph","display_name":"amjoseph","email":"adam@westernsemico.com","username":"amjoseph"},"change_message_id":"b74e61736b9934dd0a3605daf04c2bb97a630a9f","unresolved":false,"context_lines":[{"line_number":140,"context_line":""},{"line_number":141,"context_line":"Frames request suspension by re-enqueueing *themselves* through VM helper"},{"line_number":142,"context_line":"methods, and then leaving the frame they want to run *on top* of themselves in"},{"line_number":143,"context_line":"the frame stack before yielding control back to the outer loop."},{"line_number":144,"context_line":""},{"line_number":145,"context_line":"The inner control loops inform the outer loops about whether the frame has"},{"line_number":146,"context_line":"been *completed* or *suspended* by returning a boolean."}],"source_content_type":"text/x-markdown","patch_set":9,"id":"964c34e7_6ca816fe","line":143,"updated":"2023-03-08 11:58:34.000000000","message":"This is much cleaner and easier to understand than the trampoline continuation mechanism.","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000066,"name":"Adam Joseph","display_name":"amjoseph","email":"adam@westernsemico.com","username":"amjoseph"},"change_message_id":"b74e61736b9934dd0a3605daf04c2bb97a630a9f","unresolved":true,"context_lines":[{"line_number":145,"context_line":"The inner control loops inform the outer loops about whether the frame has"},{"line_number":146,"context_line":"been *completed* or *suspended* by returning a boolean."},{"line_number":147,"context_line":""},{"line_number":148,"context_line":"### Inner call frame loop"},{"line_number":149,"context_line":""},{"line_number":150,"context_line":"The inner call frame loop drives the execution of some Tvix bytecode by"},{"line_number":151,"context_line":"continously looking at the next instruction to execute, and dispatching to the"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"2a209eee_399f0457","line":148,"updated":"2023-03-08 11:58:34.000000000","message":"Wait, in the diagram, neither of the inner loops are labeled \"call frame loop\".\n\nOne is labeled \"inner bytecode loop\" and the other is labeled \"inner generator loop\".  I think you mean the first one?","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000001,"name":"tazjin","email":"tazjin@tvl.su","username":"tazjin"},"change_message_id":"46c62dd5b97259d624d775d71de7976fc2548064","unresolved":false,"context_lines":[{"line_number":145,"context_line":"The inner control loops inform the outer loops about whether the frame has"},{"line_number":146,"context_line":"been *completed* or *suspended* by returning a boolean."},{"line_number":147,"context_line":""},{"line_number":148,"context_line":"### Inner call frame loop"},{"line_number":149,"context_line":""},{"line_number":150,"context_line":"The inner call frame loop drives the execution of some Tvix bytecode by"},{"line_number":151,"context_line":"continously looking at the next instruction to execute, and dispatching to the"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"be9665fe_ddb68117","line":148,"in_reply_to":"2a209eee_399f0457","updated":"2023-03-08 12:44:36.000000000","message":"You\u0027re right, I\u0027ve updated all the terminology to be consistent with \"bytecode frame\" and \"inner bytecode loop\".","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000066,"name":"Adam Joseph","display_name":"amjoseph","email":"adam@westernsemico.com","username":"amjoseph"},"change_message_id":"b74e61736b9934dd0a3605daf04c2bb97a630a9f","unresolved":true,"context_lines":[{"line_number":180,"context_line":"With this refactoring, the compiler now emits a special `OpReturn` instruction"},{"line_number":181,"context_line":"at the end of bytecode chunks. This is a signal to the runtime that the chunk"},{"line_number":182,"context_line":"has completed and that its current value should be returned, without having to"},{"line_number":183,"context_line":"perform instruction pointer arithmetic."},{"line_number":184,"context_line":""},{"line_number":185,"context_line":"When `OpReturn` is encountered, the inner call frame loop returns control to"},{"line_number":186,"context_line":"the outer loop and informs it (by returning `true`) that the call frame has"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"a7e8242d_d6384e17","line":183,"updated":"2023-03-08 11:58:34.000000000","message":"Is it possible for one bytecode function to invoke another bytecode function (which is in a different chunk) without leaving the inner call frame loop?","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000066,"name":"Adam Joseph","display_name":"amjoseph","email":"adam@westernsemico.com","username":"amjoseph"},"change_message_id":"ce949d80d23368ad8ecc5651a287e330d43c9421","unresolved":true,"context_lines":[{"line_number":180,"context_line":"With this refactoring, the compiler now emits a special `OpReturn` instruction"},{"line_number":181,"context_line":"at the end of bytecode chunks. This is a signal to the runtime that the chunk"},{"line_number":182,"context_line":"has completed and that its current value should be returned, without having to"},{"line_number":183,"context_line":"perform instruction pointer arithmetic."},{"line_number":184,"context_line":""},{"line_number":185,"context_line":"When `OpReturn` is encountered, the inner call frame loop returns control to"},{"line_number":186,"context_line":"the outer loop and informs it (by returning `true`) that the call frame has"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"eedfeb06_b9235b76","line":183,"in_reply_to":"08411ccb_706ecc60","updated":"2023-03-11 11:09:11.000000000","message":"\u003e all function applications are actually tail calls \n\nI disagree: https://cl.tvl.fyi/c/depot/+/8147/comments/f53dd5d7_2ea6f4da","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000066,"name":"Adam Joseph","display_name":"amjoseph","email":"adam@westernsemico.com","username":"amjoseph"},"change_message_id":"e6396d9a300c62c6acd32ed7e6d27bf3ac434e4d","unresolved":false,"context_lines":[{"line_number":180,"context_line":"With this refactoring, the compiler now emits a special `OpReturn` instruction"},{"line_number":181,"context_line":"at the end of bytecode chunks. This is a signal to the runtime that the chunk"},{"line_number":182,"context_line":"has completed and that its current value should be returned, without having to"},{"line_number":183,"context_line":"perform instruction pointer arithmetic."},{"line_number":184,"context_line":""},{"line_number":185,"context_line":"When `OpReturn` is encountered, the inner call frame loop returns control to"},{"line_number":186,"context_line":"the outer loop and informs it (by returning `true`) that the call frame has"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"6326410a_8fbda31a","line":183,"in_reply_to":"a209d8b7_7fddc681","updated":"2023-03-13 14:48:12.000000000","message":"Done","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000001,"name":"tazjin","email":"tazjin@tvl.su","username":"tazjin"},"change_message_id":"46c62dd5b97259d624d775d71de7976fc2548064","unresolved":true,"context_lines":[{"line_number":180,"context_line":"With this refactoring, the compiler now emits a special `OpReturn` instruction"},{"line_number":181,"context_line":"at the end of bytecode chunks. This is a signal to the runtime that the chunk"},{"line_number":182,"context_line":"has completed and that its current value should be returned, without having to"},{"line_number":183,"context_line":"perform instruction pointer arithmetic."},{"line_number":184,"context_line":""},{"line_number":185,"context_line":"When `OpReturn` is encountered, the inner call frame loop returns control to"},{"line_number":186,"context_line":"the outer loop and informs it (by returning `true`) that the call frame has"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"08411ccb_706ecc60","line":183,"in_reply_to":"a7e8242d_d6384e17","updated":"2023-03-08 12:44:36.000000000","message":"No, but for a slightly different reason (see cl/8147). Since function applications are thunked, *all* function applications are actually tail calls when invoked from bytecode (the only function applications that are not are those invoked from builtins).\n\nWhat this means is that when a call is encountered, the current frame is always being left anyways (and currently actually removed in favour of the new one). We *could* keep them around for better traces in error reporting, but that hasn\u0027t been a focus yet.","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000001,"name":"tazjin","email":"tazjin@tvl.su","username":"tazjin"},"change_message_id":"8a789469a722f1664f74061bf90a38f8219503ba","unresolved":true,"context_lines":[{"line_number":180,"context_line":"With this refactoring, the compiler now emits a special `OpReturn` instruction"},{"line_number":181,"context_line":"at the end of bytecode chunks. This is a signal to the runtime that the chunk"},{"line_number":182,"context_line":"has completed and that its current value should be returned, without having to"},{"line_number":183,"context_line":"perform instruction pointer arithmetic."},{"line_number":184,"context_line":""},{"line_number":185,"context_line":"When `OpReturn` is encountered, the inner call frame loop returns control to"},{"line_number":186,"context_line":"the outer loop and informs it (by returning `true`) that the call frame has"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"a209d8b7_7fddc681","line":183,"in_reply_to":"eedfeb06_b9235b76","updated":"2023-03-11 13:24:12.000000000","message":"What I meant would be more precisely expressed as \"all function applications are theoretically tail-calls, as they occur in the last meaningful instruction of their code chunk, as all function applications are thunked\". Replying on the other CL about the function name 😊","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000066,"name":"Adam Joseph","display_name":"amjoseph","email":"adam@westernsemico.com","username":"amjoseph"},"change_message_id":"b74e61736b9934dd0a3605daf04c2bb97a630a9f","unresolved":true,"context_lines":[{"line_number":194,"context_line":"### Inner generator loop"},{"line_number":195,"context_line":""},{"line_number":196,"context_line":"The inner generator loop is responsible for driving the execution of a"},{"line_number":197,"context_line":"generator frame by continously calling [`Gen::resume`][]."},{"line_number":198,"context_line":""},{"line_number":199,"context_line":"```"},{"line_number":200,"context_line":"   ┏━━━━━━━━━━━━━┓       [Done]"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"f1c9dd2b_4f6b00df","line":197,"updated":"2023-03-08 11:58:34.000000000","message":"This is sort of a tautology...\n\nIf you want to keep the name \"generator loop\" please state clearly something like \"the purpose of the generator loop is to execute builtins\" or something like that.","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000001,"name":"tazjin","email":"tazjin@tvl.su","username":"tazjin"},"change_message_id":"8a789469a722f1664f74061bf90a38f8219503ba","unresolved":true,"context_lines":[{"line_number":194,"context_line":"### Inner generator loop"},{"line_number":195,"context_line":""},{"line_number":196,"context_line":"The inner generator loop is responsible for driving the execution of a"},{"line_number":197,"context_line":"generator frame by continously calling [`Gen::resume`][]."},{"line_number":198,"context_line":""},{"line_number":199,"context_line":"```"},{"line_number":200,"context_line":"   ┏━━━━━━━━━━━━━┓       [Done]"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"fb4e08d1_720f7adb","line":197,"in_reply_to":"0fff7400_6cae21b5","updated":"2023-03-11 13:24:12.000000000","message":"Hm, the requests do not go from the generators to bytecode chunks, though. They go to the VM. In a sense, the generators ... generate ... requests that are consumed by the *VM*. This also makes me hesitant about your suggested naming of the request/response function types, as it\u0027s totally possible (and in fact frequently occurs) that a generator requests something that another generator produces, in which case `NativeToBytecode` is incorrect - there\u0027s no bytecode involved.\n\nThis all fundamentally revolves around the VM in my head, so if we go down that naming route it should be something closer to `NativeToVM`, `VMToNative`, but these also sound strange to me because, well, the VM is in fact written in native Rust.","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000001,"name":"tazjin","email":"tazjin@tvl.su","username":"tazjin"},"change_message_id":"47313bbce418dd8d1fe47d6603984f7ef4ccb994","unresolved":false,"context_lines":[{"line_number":194,"context_line":"### Inner generator loop"},{"line_number":195,"context_line":""},{"line_number":196,"context_line":"The inner generator loop is responsible for driving the execution of a"},{"line_number":197,"context_line":"generator frame by continously calling [`Gen::resume`][]."},{"line_number":198,"context_line":""},{"line_number":199,"context_line":"```"},{"line_number":200,"context_line":"   ┏━━━━━━━━━━━━━┓       [Done]"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"5572f1f2_93a07848","line":197,"in_reply_to":"7d0df0f0_f12315f9","updated":"2023-03-14 09:22:30.000000000","message":"Done","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000066,"name":"Adam Joseph","display_name":"amjoseph","email":"adam@westernsemico.com","username":"amjoseph"},"change_message_id":"ce949d80d23368ad8ecc5651a287e330d43c9421","unresolved":true,"context_lines":[{"line_number":194,"context_line":"### Inner generator loop"},{"line_number":195,"context_line":""},{"line_number":196,"context_line":"The inner generator loop is responsible for driving the execution of a"},{"line_number":197,"context_line":"generator frame by continously calling [`Gen::resume`][]."},{"line_number":198,"context_line":""},{"line_number":199,"context_line":"```"},{"line_number":200,"context_line":"   ┏━━━━━━━━━━━━━┓       [Done]"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"0fff7400_6cae21b5","line":197,"in_reply_to":"d11e0e70_1785de4c","updated":"2023-03-11 11:09:11.000000000","message":"Yeah, I get that.\n\nI think the `genawaiter` crate\u0027s terminology is a really bad fit for the way tvix is using it.\n\n`genawaiter` is meant for situations where you very obviously have a producer and a consumer; the consumer pulls data out of the producer and the producer is otherwise passive and never makes any requests to the consumer.\n\nTvix is implementing true coroutines, where neither the bytecode side nor the rust side is truly \"in charge\".  Bytecode can call `builtins` (which are rust code) and rust can call bytecode (e.g. `builtins.sort` calling the comparison function that was passed to it).\n\nIf you\u0027ve found the `genawaiter` codebase meets your needs great, keep using it, but I urge you to totally abandon their terminology.  Otherwise people reading your code have to memorize \"generator means rust code\", \"resume means calling a builtin\", etc etc.  It\u0027s not like math or physics where you can skip the memorization and re-derive these things from first principles when you take the exam... the names are now arbitrary because you don\u0027t have a generator/consumer anymore, you have two coroutines.","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000001,"name":"tazjin","email":"tazjin@tvl.su","username":"tazjin"},"change_message_id":"f96697bfa09af80f4a44e3a2df70c44b57a24f6d","unresolved":true,"context_lines":[{"line_number":194,"context_line":"### Inner generator loop"},{"line_number":195,"context_line":""},{"line_number":196,"context_line":"The inner generator loop is responsible for driving the execution of a"},{"line_number":197,"context_line":"generator frame by continously calling [`Gen::resume`][]."},{"line_number":198,"context_line":""},{"line_number":199,"context_line":"```"},{"line_number":200,"context_line":"   ┏━━━━━━━━━━━━━┓       [Done]"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"7d0df0f0_f12315f9","line":197,"in_reply_to":"e35f4f53_6a1abce6","updated":"2023-03-13 21:50:04.000000000","message":"Done in cl/8291, please approve that. I\u0027ll have to update the link to the type in this document after that is submitted.","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000066,"name":"Adam Joseph","display_name":"amjoseph","email":"adam@westernsemico.com","username":"amjoseph"},"change_message_id":"e6396d9a300c62c6acd32ed7e6d27bf3ac434e4d","unresolved":true,"context_lines":[{"line_number":194,"context_line":"### Inner generator loop"},{"line_number":195,"context_line":""},{"line_number":196,"context_line":"The inner generator loop is responsible for driving the execution of a"},{"line_number":197,"context_line":"generator frame by continously calling [`Gen::resume`][]."},{"line_number":198,"context_line":""},{"line_number":199,"context_line":"```"},{"line_number":200,"context_line":"   ┏━━━━━━━━━━━━━┓       [Done]"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"e35f4f53_6a1abce6","line":197,"in_reply_to":"ec89d21e_5cdb0e26","updated":"2023-03-13 14:48:12.000000000","message":"\u003e GeneratorMessage/NativeMessage - a message from a generator/native code\n\nThat sounds great.\n\n\u003e VMMessage - a message from the VM\n\nThe `VMRequest` (request *to* the VM) / `VMResponse` (response *from* the VM) would also be great.\n\nEither one.\n\nMark this resolved when you update it.","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000001,"name":"tazjin","email":"tazjin@tvl.su","username":"tazjin"},"change_message_id":"46c62dd5b97259d624d775d71de7976fc2548064","unresolved":true,"context_lines":[{"line_number":194,"context_line":"### Inner generator loop"},{"line_number":195,"context_line":""},{"line_number":196,"context_line":"The inner generator loop is responsible for driving the execution of a"},{"line_number":197,"context_line":"generator frame by continously calling [`Gen::resume`][]."},{"line_number":198,"context_line":""},{"line_number":199,"context_line":"```"},{"line_number":200,"context_line":"   ┏━━━━━━━━━━━━━┓       [Done]"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"d11e0e70_1785de4c","line":197,"in_reply_to":"f1c9dd2b_4f6b00df","updated":"2023-03-08 12:44:36.000000000","message":"It\u0027s not just for builtins, there are several things that it does.\n\nThere is literally a loop which (somewhat simplified) does this:\n\n  loop {\n    match gen.resume_with(msg) { \n      /* ... */\n    }\n  }\n  \nIt is this loop which \"drives\" the execution of the generator. Based on what the generator returns (either \"Yielded\", which means it\u0027s requesting something from the VM and the async state machine is suspended at that point, or \"Done\", which means it has run to completion and its result goes on the stack).\n\nI\u0027ve rephrased the sentence a little bit to try and make this clearer, as I think the missing link was that behaviour described in my last paragraph.","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000001,"name":"tazjin","email":"tazjin@tvl.su","username":"tazjin"},"change_message_id":"96281657442ae36d259721122180249203c0a4f2","unresolved":true,"context_lines":[{"line_number":194,"context_line":"### Inner generator loop"},{"line_number":195,"context_line":""},{"line_number":196,"context_line":"The inner generator loop is responsible for driving the execution of a"},{"line_number":197,"context_line":"generator frame by continously calling [`Gen::resume`][]."},{"line_number":198,"context_line":""},{"line_number":199,"context_line":"```"},{"line_number":200,"context_line":"   ┏━━━━━━━━━━━━━┓       [Done]"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"ec89d21e_5cdb0e26","line":197,"in_reply_to":"fb4e08d1_720f7adb","updated":"2023-03-11 20:23:16.000000000","message":"What about naming them based on who produces them, so either a generator/native code or the VM?\n\nGeneratorMessage/NativeMessage - a message from a generator/native code\n\nVMMessage - a message from the VM","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000066,"name":"Adam Joseph","display_name":"amjoseph","email":"adam@westernsemico.com","username":"amjoseph"},"change_message_id":"b74e61736b9934dd0a3605daf04c2bb97a630a9f","unresolved":true,"context_lines":[{"line_number":206,"context_line":"            ╭───────┤ inspect generator response │"},{"line_number":207,"context_line":"            │       ╰──────────────────┬─────────╯"},{"line_number":208,"context_line":"   ┏━━━━━━━━┷━━━━━━━━┓                 │"},{"line_number":209,"context_line":"──►┃ gen.resume(msg) ┃                 │[Yielded]"},{"line_number":210,"context_line":"   ┗━━━━━━━━━━━━━━━━━┛                 │"},{"line_number":211,"context_line":"            ▲                     ╭────┴─────╮"},{"line_number":212,"context_line":"            │          [yes]      │  inline  │"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"f7c93ed5_3b5e396e","line":209,"updated":"2023-03-08 11:58:34.000000000","message":"Nitpick: the diagram would be easier to understand if you reflect the inner loop about the vertical axis.\n\nThat way it looks like \"the generator\" is off to our right and \"the bytecode loop\" (and the rest of the VM) is off to our left.  Like sitting between two people at a table and turning your attention from one of them to the other (which is basically what tvix is doing here).","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000066,"name":"Adam Joseph","display_name":"amjoseph","email":"adam@westernsemico.com","username":"amjoseph"},"change_message_id":"ce949d80d23368ad8ecc5651a287e330d43c9421","unresolved":true,"context_lines":[{"line_number":206,"context_line":"            ╭───────┤ inspect generator response │"},{"line_number":207,"context_line":"            │       ╰──────────────────┬─────────╯"},{"line_number":208,"context_line":"   ┏━━━━━━━━┷━━━━━━━━┓                 │"},{"line_number":209,"context_line":"──►┃ gen.resume(msg) ┃                 │[Yielded]"},{"line_number":210,"context_line":"   ┗━━━━━━━━━━━━━━━━━┛                 │"},{"line_number":211,"context_line":"            ▲                     ╭────┴─────╮"},{"line_number":212,"context_line":"            │          [yes]      │  inline  │"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"c25f8809_477e0bbf","line":209,"in_reply_to":"a6ee57bb_e676206a","updated":"2023-03-11 11:09:11.000000000","message":"cl/8252","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000001,"name":"tazjin","email":"tazjin@tvl.su","username":"tazjin"},"change_message_id":"8a789469a722f1664f74061bf90a38f8219503ba","unresolved":false,"context_lines":[{"line_number":206,"context_line":"            ╭───────┤ inspect generator response │"},{"line_number":207,"context_line":"            │       ╰──────────────────┬─────────╯"},{"line_number":208,"context_line":"   ┏━━━━━━━━┷━━━━━━━━┓                 │"},{"line_number":209,"context_line":"──►┃ gen.resume(msg) ┃                 │[Yielded]"},{"line_number":210,"context_line":"   ┗━━━━━━━━━━━━━━━━━┛                 │"},{"line_number":211,"context_line":"            ▲                     ╭────┴─────╮"},{"line_number":212,"context_line":"            │          [yes]      │  inline  │"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"d1dd85fb_393b750c","line":209,"in_reply_to":"c25f8809_477e0bbf","updated":"2023-03-11 13:24:12.000000000","message":"Aha! Yeah, sure, lets do that. I\u0027ll approve your CL and we can merge it once this one is submitted.","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000001,"name":"tazjin","email":"tazjin@tvl.su","username":"tazjin"},"change_message_id":"46c62dd5b97259d624d775d71de7976fc2548064","unresolved":true,"context_lines":[{"line_number":206,"context_line":"            ╭───────┤ inspect generator response │"},{"line_number":207,"context_line":"            │       ╰──────────────────┬─────────╯"},{"line_number":208,"context_line":"   ┏━━━━━━━━┷━━━━━━━━┓                 │"},{"line_number":209,"context_line":"──►┃ gen.resume(msg) ┃                 │[Yielded]"},{"line_number":210,"context_line":"   ┗━━━━━━━━━━━━━━━━━┛                 │"},{"line_number":211,"context_line":"            ▲                     ╭────┴─────╮"},{"line_number":212,"context_line":"            │          [yes]      │  inline  │"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"a6ee57bb_e676206a","line":209,"in_reply_to":"f7c93ed5_3b5e396e","updated":"2023-03-08 12:44:36.000000000","message":"I need to sketch this on paper first, haven\u0027t quite understood exactly what you\u0027re visualising yet :) \n\nCurrently the left side is where \"all the rest is\", so the outer arrows \"come from\" and \"go\" there.","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000066,"name":"Adam Joseph","display_name":"amjoseph","email":"adam@westernsemico.com","username":"amjoseph"},"change_message_id":"b74e61736b9934dd0a3605daf04c2bb97a630a9f","unresolved":true,"context_lines":[{"line_number":218,"context_line":"```"},{"line_number":219,"context_line":""},{"line_number":220,"context_line":"On each execution of a generator frame, `resume_with` is called with a"},{"line_number":221,"context_line":"[`GeneratorResponse`][] (i.e. a message *from* the VM *to* the generator). For"},{"line_number":222,"context_line":"a newly created generator, the initial message is just `Empty`."},{"line_number":223,"context_line":""},{"line_number":224,"context_line":"A generator may then respond by signaling that it has finished execution"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"d8ac993b_0346c71d","line":221,"updated":"2023-03-08 11:58:34.000000000","message":"\u003e a message *from* the VM *to* the generator\n\nI find this really confusing and hard to keep straight.  I\u0027m not saying it\u0027s incorrect, just confusing.\n\nThe generator exists in order to execute builtins.  Is it legitimate to call this the \"builtins loop\"?  Does the generator ever do anything other than execute (native, Rust) code which is part of the implementation of some builtin?","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000001,"name":"tazjin","email":"tazjin@tvl.su","username":"tazjin"},"change_message_id":"df07658b4b79214ea6541cd86bc0f0830b4773b3","unresolved":false,"context_lines":[{"line_number":218,"context_line":"```"},{"line_number":219,"context_line":""},{"line_number":220,"context_line":"On each execution of a generator frame, `resume_with` is called with a"},{"line_number":221,"context_line":"[`GeneratorResponse`][] (i.e. a message *from* the VM *to* the generator). For"},{"line_number":222,"context_line":"a newly created generator, the initial message is just `Empty`."},{"line_number":223,"context_line":""},{"line_number":224,"context_line":"A generator may then respond by signaling that it has finished execution"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"2d055b6d_2ddd4569","line":221,"in_reply_to":"2e08fda0_d5fd6192","updated":"2023-03-13 21:49:42.000000000","message":"Ack","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000066,"name":"Adam Joseph","display_name":"amjoseph","email":"adam@westernsemico.com","username":"amjoseph"},"change_message_id":"ce949d80d23368ad8ecc5651a287e330d43c9421","unresolved":true,"context_lines":[{"line_number":218,"context_line":"```"},{"line_number":219,"context_line":""},{"line_number":220,"context_line":"On each execution of a generator frame, `resume_with` is called with a"},{"line_number":221,"context_line":"[`GeneratorResponse`][] (i.e. a message *from* the VM *to* the generator). For"},{"line_number":222,"context_line":"a newly created generator, the initial message is just `Empty`."},{"line_number":223,"context_line":""},{"line_number":224,"context_line":"A generator may then respond by signaling that it has finished execution"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"90679fa3_bb0ccb91","line":221,"in_reply_to":"6dfbe3ce_702c00db","updated":"2023-03-11 11:09:11.000000000","message":"\u003e It would be correct to say that the generators always execute some native Rust code, which is part of the evaluation implementation. \n\nOkay that makes sense.\n\n\u003e \"GeneratorRequest\" (requesting something from the VM), and the VM sends it a \"GeneratorResponse\" (returning the result to the VM).\n\nI keep stumbling over this too.  GeneratorRequest is a request *from* the generator not a request *to* the generator?  Isn\u0027t that sort of the opposite of the convention?  Like how HTTPRequest is generally a request *to* the HTTP server?","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000001,"name":"tazjin","email":"tazjin@tvl.su","username":"tazjin"},"change_message_id":"8a789469a722f1664f74061bf90a38f8219503ba","unresolved":true,"context_lines":[{"line_number":218,"context_line":"```"},{"line_number":219,"context_line":""},{"line_number":220,"context_line":"On each execution of a generator frame, `resume_with` is called with a"},{"line_number":221,"context_line":"[`GeneratorResponse`][] (i.e. a message *from* the VM *to* the generator). For"},{"line_number":222,"context_line":"a newly created generator, the initial message is just `Empty`."},{"line_number":223,"context_line":""},{"line_number":224,"context_line":"A generator may then respond by signaling that it has finished execution"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"c4805c79_2f2ee5c5","line":221,"in_reply_to":"90679fa3_bb0ccb91","updated":"2023-03-11 13:24:12.000000000","message":"In some sense the VM is a \"server\" here that receives \"requests\" and returns \"responses\", whether those involve running other frames (native or bytecode), or doing something in the VM, is kind of irrelevant for the thing doing the requesting (the generators).\n\nI think following the HTTP convention would lead us to something like `VMRequest` / `VMResponse`. Hmm, I\u0027m not sure.","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000066,"name":"Adam Joseph","display_name":"amjoseph","email":"adam@westernsemico.com","username":"amjoseph"},"change_message_id":"e6396d9a300c62c6acd32ed7e6d27bf3ac434e4d","unresolved":true,"context_lines":[{"line_number":218,"context_line":"```"},{"line_number":219,"context_line":""},{"line_number":220,"context_line":"On each execution of a generator frame, `resume_with` is called with a"},{"line_number":221,"context_line":"[`GeneratorResponse`][] (i.e. a message *from* the VM *to* the generator). For"},{"line_number":222,"context_line":"a newly created generator, the initial message is just `Empty`."},{"line_number":223,"context_line":""},{"line_number":224,"context_line":"A generator may then respond by signaling that it has finished execution"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"2e08fda0_d5fd6192","line":221,"in_reply_to":"c4805c79_2f2ee5c5","updated":"2023-03-13 14:48:12.000000000","message":"\u003e I think following the HTTP convention would lead us to something like VMRequest / VMResponse.\n\nThat would be a big improvement.","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000001,"name":"tazjin","email":"tazjin@tvl.su","username":"tazjin"},"change_message_id":"46c62dd5b97259d624d775d71de7976fc2548064","unresolved":true,"context_lines":[{"line_number":218,"context_line":"```"},{"line_number":219,"context_line":""},{"line_number":220,"context_line":"On each execution of a generator frame, `resume_with` is called with a"},{"line_number":221,"context_line":"[`GeneratorResponse`][] (i.e. a message *from* the VM *to* the generator). For"},{"line_number":222,"context_line":"a newly created generator, the initial message is just `Empty`."},{"line_number":223,"context_line":""},{"line_number":224,"context_line":"A generator may then respond by signaling that it has finished execution"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"6dfbe3ce_702c00db","line":221,"in_reply_to":"d8ac993b_0346c71d","updated":"2023-03-08 12:44:36.000000000","message":"The confusing thing here is that the *first* time a generator is invoked after being instantiated, it is called with a no-op message, which is a constraint imposed by the API of crate that we use for the generator data structures.\n\nAfter that, there is always a ping/pong playing out between the generator and the rest of the VM, in which the generator sends a \"GeneratorRequest\" (requesting something from the VM), and the VM sends it a \"GeneratorResponse\" (returning the result to the VM).\n\nThese requests/responses (linked below in the text under the respective types) are things like \"please force this value\" or \"please import this file through the IO interface\", or in general all things where we previously had a `\u0026mut VM` (which is no longer the case anywhere - VM is now a private type).\n\nGenerators do more than just the execution of builtins. For example, forcing is also a generator because there is some native Rust code (heavily based on the one you wrote for trampolines, and my rewrite of it from cl/8012) that figures out what to do with a given thunk before just entering its bytecode loop.\n\nIt would be correct to say that the generators always execute some native Rust code, which is part of the evaluation implementation. But it\u0027s not just builtins.","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000066,"name":"Adam Joseph","display_name":"amjoseph","email":"adam@westernsemico.com","username":"amjoseph"},"change_message_id":"b74e61736b9934dd0a3605daf04c2bb97a630a9f","unresolved":true,"context_lines":[{"line_number":231,"context_line":"requests can be very simple (\"Tell me the current store path\") or more complex"},{"line_number":232,"context_line":"(\"Call this Nix function with these values\")."},{"line_number":233,"context_line":""},{"line_number":234,"context_line":"Requests are divided into two classes: Inline requests (requests that can be"},{"line_number":235,"context_line":"responded to *without* returning control to the outer loop, i.e. without"},{"line_number":236,"context_line":"executing a *different* frame), and other requests. Based on the type of"},{"line_number":237,"context_line":"request, the inner generator loop will either handle it right away and send"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"fdd3fee1_6d818cfc","line":234,"updated":"2023-03-08 11:58:34.000000000","message":"Can you give an example of an inline request?  I\u0027m having trouble thinking of one.  When a builtin needs to execute bytecode that would be an \"other request\" since it needs to leave the generator loop and move into the bytecode loop.","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000066,"name":"Adam Joseph","display_name":"amjoseph","email":"adam@westernsemico.com","username":"amjoseph"},"change_message_id":"b74e61736b9934dd0a3605daf04c2bb97a630a9f","unresolved":true,"context_lines":[{"line_number":231,"context_line":"requests can be very simple (\"Tell me the current store path\") or more complex"},{"line_number":232,"context_line":"(\"Call this Nix function with these values\")."},{"line_number":233,"context_line":""},{"line_number":234,"context_line":"Requests are divided into two classes: Inline requests (requests that can be"},{"line_number":235,"context_line":"responded to *without* returning control to the outer loop, i.e. without"},{"line_number":236,"context_line":"executing a *different* frame), and other requests. Based on the type of"},{"line_number":237,"context_line":"request, the inner generator loop will either handle it right away and send"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"0ac68224_35f9fa5b","line":234,"updated":"2023-03-08 11:58:34.000000000","message":"Suggest s/Requests/Generator requests/ for clarity.","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000001,"name":"tazjin","email":"tazjin@tvl.su","username":"tazjin"},"change_message_id":"46c62dd5b97259d624d775d71de7976fc2548064","unresolved":false,"context_lines":[{"line_number":231,"context_line":"requests can be very simple (\"Tell me the current store path\") or more complex"},{"line_number":232,"context_line":"(\"Call this Nix function with these values\")."},{"line_number":233,"context_line":""},{"line_number":234,"context_line":"Requests are divided into two classes: Inline requests (requests that can be"},{"line_number":235,"context_line":"responded to *without* returning control to the outer loop, i.e. without"},{"line_number":236,"context_line":"executing a *different* frame), and other requests. Based on the type of"},{"line_number":237,"context_line":"request, the inner generator loop will either handle it right away and send"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"12aa643c_7a86ef3e","line":234,"in_reply_to":"0ac68224_35f9fa5b","updated":"2023-03-08 12:44:36.000000000","message":"Done","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000066,"name":"Adam Joseph","display_name":"amjoseph","email":"adam@westernsemico.com","username":"amjoseph"},"change_message_id":"ce949d80d23368ad8ecc5651a287e330d43c9421","unresolved":true,"context_lines":[{"line_number":231,"context_line":"requests can be very simple (\"Tell me the current store path\") or more complex"},{"line_number":232,"context_line":"(\"Call this Nix function with these values\")."},{"line_number":233,"context_line":""},{"line_number":234,"context_line":"Requests are divided into two classes: Inline requests (requests that can be"},{"line_number":235,"context_line":"responded to *without* returning control to the outer loop, i.e. without"},{"line_number":236,"context_line":"executing a *different* frame), and other requests. Based on the type of"},{"line_number":237,"context_line":"request, the inner generator loop will either handle it right away and send"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"69d3a3a9_e26e0660","line":234,"in_reply_to":"286b1530_12cfd6a9","updated":"2023-03-11 11:09:11.000000000","message":"See https://cl.tvl.fyi/c/depot/+/8226/comment/c2cc2023_c7170f85/","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000001,"name":"tazjin","email":"tazjin@tvl.su","username":"tazjin"},"change_message_id":"8a789469a722f1664f74061bf90a38f8219503ba","unresolved":false,"context_lines":[{"line_number":231,"context_line":"requests can be very simple (\"Tell me the current store path\") or more complex"},{"line_number":232,"context_line":"(\"Call this Nix function with these values\")."},{"line_number":233,"context_line":""},{"line_number":234,"context_line":"Requests are divided into two classes: Inline requests (requests that can be"},{"line_number":235,"context_line":"responded to *without* returning control to the outer loop, i.e. without"},{"line_number":236,"context_line":"executing a *different* frame), and other requests. Based on the type of"},{"line_number":237,"context_line":"request, the inner generator loop will either handle it right away and send"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"fe5d1cbb_95f1ef52","line":234,"in_reply_to":"69d3a3a9_e26e0660","updated":"2023-03-11 13:24:12.000000000","message":"Ack","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000001,"name":"tazjin","email":"tazjin@tvl.su","username":"tazjin"},"change_message_id":"b5aa37f9837fe572164b184f28bfffa973cfdd61","unresolved":true,"context_lines":[{"line_number":231,"context_line":"requests can be very simple (\"Tell me the current store path\") or more complex"},{"line_number":232,"context_line":"(\"Call this Nix function with these values\")."},{"line_number":233,"context_line":""},{"line_number":234,"context_line":"Requests are divided into two classes: Inline requests (requests that can be"},{"line_number":235,"context_line":"responded to *without* returning control to the outer loop, i.e. without"},{"line_number":236,"context_line":"executing a *different* frame), and other requests. Based on the type of"},{"line_number":237,"context_line":"request, the inner generator loop will either handle it right away and send"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"286b1530_12cfd6a9","line":234,"in_reply_to":"9f9c162a_e4098f62","updated":"2023-03-08 12:52:08.000000000","message":"maybe the other ones are \"frame requests\" because they request execution of another frame (whether generator or bytecode)? Hmm","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000001,"name":"tazjin","email":"tazjin@tvl.su","username":"tazjin"},"change_message_id":"46c62dd5b97259d624d775d71de7976fc2548064","unresolved":true,"context_lines":[{"line_number":231,"context_line":"requests can be very simple (\"Tell me the current store path\") or more complex"},{"line_number":232,"context_line":"(\"Call this Nix function with these values\")."},{"line_number":233,"context_line":""},{"line_number":234,"context_line":"Requests are divided into two classes: Inline requests (requests that can be"},{"line_number":235,"context_line":"responded to *without* returning control to the outer loop, i.e. without"},{"line_number":236,"context_line":"executing a *different* frame), and other requests. Based on the type of"},{"line_number":237,"context_line":"request, the inner generator loop will either handle it right away and send"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"9f9c162a_e4098f62","line":234,"in_reply_to":"fdd3fee1_6d818cfc","updated":"2023-03-08 12:44:36.000000000","message":"Here are some of the inline requests we have right now:\n\n1. Stack push (used under-the-hood by a helper function to prepare the VM\u0027s stack for a function call from a generator).\n\n2. Emitting a warning during evaluation.\n\n3. Interacting with the VM\u0027s import cache (to retrieve already imported thunks when importing Nix files).\n\n4. Interacting with the EvalIO interface (I think this didn\u0027t exist last time you were around: It\u0027s the bridge between tvix-eval and things like the store, so stuff like \"does this path exist?\" or \"read this directory!\").\n\nBasically it\u0027s operations that previously required a `\u0026mut VM`, which the generator loop (which is part of the VM implementation and has access to its internals) can do.","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000066,"name":"Adam Joseph","display_name":"amjoseph","email":"adam@westernsemico.com","username":"amjoseph"},"change_message_id":"b74e61736b9934dd0a3605daf04c2bb97a630a9f","unresolved":true,"context_lines":[{"line_number":233,"context_line":""},{"line_number":234,"context_line":"Requests are divided into two classes: Inline requests (requests that can be"},{"line_number":235,"context_line":"responded to *without* returning control to the outer loop, i.e. without"},{"line_number":236,"context_line":"executing a *different* frame), and other requests. Based on the type of"},{"line_number":237,"context_line":"request, the inner generator loop will either handle it right away and send"},{"line_number":238,"context_line":"the response in a new `resume_with` call, or return `false` to the outer"},{"line_number":239,"context_line":"generator loop after setting up the frame stack."}],"source_content_type":"text/x-markdown","patch_set":9,"id":"c2cc2023_c7170f85","line":236,"range":{"start_line":236,"start_character":36,"end_line":236,"end_character":50},"updated":"2023-03-08 11:58:34.000000000","message":"We need a better name than \"other requests\".","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000066,"name":"Adam Joseph","display_name":"amjoseph","email":"adam@westernsemico.com","username":"amjoseph"},"change_message_id":"ce949d80d23368ad8ecc5651a287e330d43c9421","unresolved":true,"context_lines":[{"line_number":233,"context_line":""},{"line_number":234,"context_line":"Requests are divided into two classes: Inline requests (requests that can be"},{"line_number":235,"context_line":"responded to *without* returning control to the outer loop, i.e. without"},{"line_number":236,"context_line":"executing a *different* frame), and other requests. Based on the type of"},{"line_number":237,"context_line":"request, the inner generator loop will either handle it right away and send"},{"line_number":238,"context_line":"the response in a new `resume_with` call, or return `false` to the outer"},{"line_number":239,"context_line":"generator loop after setting up the frame stack."}],"source_content_type":"text/x-markdown","patch_set":9,"id":"a89af6c5_9a78f66c","line":236,"range":{"start_line":236,"start_character":36,"end_line":236,"end_character":50},"in_reply_to":"8e3449d5_e85be0c7","updated":"2023-03-11 11:09:11.000000000","message":"How about single-frame request and multi-frame request?","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000001,"name":"tazjin","email":"tazjin@tvl.su","username":"tazjin"},"change_message_id":"8a789469a722f1664f74061bf90a38f8219503ba","unresolved":true,"context_lines":[{"line_number":233,"context_line":""},{"line_number":234,"context_line":"Requests are divided into two classes: Inline requests (requests that can be"},{"line_number":235,"context_line":"responded to *without* returning control to the outer loop, i.e. without"},{"line_number":236,"context_line":"executing a *different* frame), and other requests. Based on the type of"},{"line_number":237,"context_line":"request, the inner generator loop will either handle it right away and send"},{"line_number":238,"context_line":"the response in a new `resume_with` call, or return `false` to the outer"},{"line_number":239,"context_line":"generator loop after setting up the frame stack."}],"source_content_type":"text/x-markdown","patch_set":9,"id":"bc5bf660_8ef269c6","line":236,"range":{"start_line":236,"start_character":36,"end_line":236,"end_character":50},"in_reply_to":"a89af6c5_9a78f66c","updated":"2023-03-11 13:24:12.000000000","message":"That\u0027s good. How about s/single/same, so we get same-frame request and multi-frame request?","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000001,"name":"tazjin","email":"tazjin@tvl.su","username":"tazjin"},"change_message_id":"df07658b4b79214ea6541cd86bc0f0830b4773b3","unresolved":false,"context_lines":[{"line_number":233,"context_line":""},{"line_number":234,"context_line":"Requests are divided into two classes: Inline requests (requests that can be"},{"line_number":235,"context_line":"responded to *without* returning control to the outer loop, i.e. without"},{"line_number":236,"context_line":"executing a *different* frame), and other requests. Based on the type of"},{"line_number":237,"context_line":"request, the inner generator loop will either handle it right away and send"},{"line_number":238,"context_line":"the response in a new `resume_with` call, or return `false` to the outer"},{"line_number":239,"context_line":"generator loop after setting up the frame stack."}],"source_content_type":"text/x-markdown","patch_set":9,"id":"f72aafd2_f885378b","line":236,"range":{"start_line":236,"start_character":36,"end_line":236,"end_character":50},"in_reply_to":"b28c5126_08a00945","updated":"2023-03-13 21:49:42.000000000","message":"Done","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000066,"name":"Adam Joseph","display_name":"amjoseph","email":"adam@westernsemico.com","username":"amjoseph"},"change_message_id":"e6396d9a300c62c6acd32ed7e6d27bf3ac434e4d","unresolved":true,"context_lines":[{"line_number":233,"context_line":""},{"line_number":234,"context_line":"Requests are divided into two classes: Inline requests (requests that can be"},{"line_number":235,"context_line":"responded to *without* returning control to the outer loop, i.e. without"},{"line_number":236,"context_line":"executing a *different* frame), and other requests. Based on the type of"},{"line_number":237,"context_line":"request, the inner generator loop will either handle it right away and send"},{"line_number":238,"context_line":"the response in a new `resume_with` call, or return `false` to the outer"},{"line_number":239,"context_line":"generator loop after setting up the frame stack."}],"source_content_type":"text/x-markdown","patch_set":9,"id":"b28c5126_08a00945","line":236,"range":{"start_line":236,"start_character":36,"end_line":236,"end_character":50},"in_reply_to":"bc5bf660_8ef269c6","updated":"2023-03-13 14:48:12.000000000","message":"Sounds great.","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000001,"name":"tazjin","email":"tazjin@tvl.su","username":"tazjin"},"change_message_id":"46c62dd5b97259d624d775d71de7976fc2548064","unresolved":true,"context_lines":[{"line_number":233,"context_line":""},{"line_number":234,"context_line":"Requests are divided into two classes: Inline requests (requests that can be"},{"line_number":235,"context_line":"responded to *without* returning control to the outer loop, i.e. without"},{"line_number":236,"context_line":"executing a *different* frame), and other requests. Based on the type of"},{"line_number":237,"context_line":"request, the inner generator loop will either handle it right away and send"},{"line_number":238,"context_line":"the response in a new `resume_with` call, or return `false` to the outer"},{"line_number":239,"context_line":"generator loop after setting up the frame stack."}],"source_content_type":"text/x-markdown","patch_set":9,"id":"8e3449d5_e85be0c7","line":236,"range":{"start_line":236,"start_character":36,"end_line":236,"end_character":50},"in_reply_to":"c2cc2023_c7170f85","updated":"2023-03-08 12:44:36.000000000","message":"Suggestions welcome! They\u0027re \"more complex\" than inline requests, so maybe something like \"complex requests\"? Those would be stuff like \"force a value\", \"call a Nix function\", \"coerce value to string\" and so on.","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000066,"name":"Adam Joseph","display_name":"amjoseph","email":"adam@westernsemico.com","username":"amjoseph"},"change_message_id":"b74e61736b9934dd0a3605daf04c2bb97a630a9f","unresolved":true,"context_lines":[{"line_number":303,"context_line":"   packages (e.g. for `//tvix/serde`), and its profile should be as slim as"},{"line_number":304,"context_line":"   possible."},{"line_number":305,"context_line":""},{"line_number":306,"context_line":"3. ... ?"},{"line_number":307,"context_line":""},{"line_number":308,"context_line":"[`genawaiter`]: https://docs.rs/genawaiter/"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"c92ad470_84e78f87","line":306,"updated":"2023-03-08 11:58:34.000000000","message":"3. Convince the Rust devs that Rust needs a way to guarantee constant-stack-depth tail calls (most likely a `tailcall` keyword).\n\nObviously this isn\u0027t going to happen, but the point here is that this is really all about not overflowing the stack.  Other interpreters (Boa) written in Rust have hit this problem too.","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000001,"name":"tazjin","email":"tazjin@tvl.su","username":"tazjin"},"change_message_id":"46c62dd5b97259d624d775d71de7976fc2548064","unresolved":false,"context_lines":[{"line_number":303,"context_line":"   packages (e.g. for `//tvix/serde`), and its profile should be as slim as"},{"line_number":304,"context_line":"   possible."},{"line_number":305,"context_line":""},{"line_number":306,"context_line":"3. ... ?"},{"line_number":307,"context_line":""},{"line_number":308,"context_line":"[`genawaiter`]: https://docs.rs/genawaiter/"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"d1169a64_58b73cd5","line":306,"in_reply_to":"c92ad470_84e78f87","updated":"2023-03-08 12:44:36.000000000","message":"Added it for completeness sake, though I\u0027m not convinced we can actually implement everything here through tail calls ;)","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"},{"author":{"_account_id":1000066,"name":"Adam Joseph","display_name":"amjoseph","email":"adam@westernsemico.com","username":"amjoseph"},"change_message_id":"ce949d80d23368ad8ecc5651a287e330d43c9421","unresolved":false,"context_lines":[{"line_number":303,"context_line":"   packages (e.g. for `//tvix/serde`), and its profile should be as slim as"},{"line_number":304,"context_line":"   possible."},{"line_number":305,"context_line":""},{"line_number":306,"context_line":"3. ... ?"},{"line_number":307,"context_line":""},{"line_number":308,"context_line":"[`genawaiter`]: https://docs.rs/genawaiter/"}],"source_content_type":"text/x-markdown","patch_set":9,"id":"6818b149_96fb1e11","line":306,"in_reply_to":"d1169a64_58b73cd5","updated":"2023-03-11 11:09:11.000000000","message":"Agreed.","commit_id":"b98fd5fd74481c46c91d3d309679138153b5e8c9"}]}