## Raw notes - How to pause an agent or prompt for input, or sub-refine when there's conflicting results on a step. e.g. results from a web search are inconclusive because it's new and uncertain information. e.g. who is a public figure dating right after a breakup or a when they just start dating someone new - need to account for conflicting evidence, surface the uncertainty, and provide a conflict resolution strategy. - carry on along multiple fronts - if I assume X is true... If I assume Y is true... - e.g. the LangChain serpapi tool has a hard coded strategy for interpreting the google search results. What I really want is the ability to have a dynamic strategy where the contents returned makes up evidence. - it might even be forking multiple agents to explore hypotheses on multiple fronts - Similar to conflicting, results were too broad, you want a search term refinement loop - the best search term at one moment in time might not always be the best, you might - Need adaptive timeout and degraded experience fallbacks - e.g. web search + index search, index search doesn't return in time. When do you wait, when do you proceed. Can you proceed and augment results later? - visualizing the embedding space of a step could be a useful icon - meta reasoning. e.g. an agent that can reason about the capability of all available tools and ask for a tool. "If I had access to FOO I could solve BLAH by doing THING" - [[Theorem prover tactic|Tactics]] and [[Tactical|tacticals]] - Tactics ~ a generated chain (to be parameterized)? - tacticals ~ a meta chain template? - Open fronts and convergence similar to unproven branches or holes in an [[Automated Theorem Prover]]. - Remembering previously generated chains, what worked and didn't (and why) so you can crib that as a starting point in the future but if that overly refined strategy fails, you can back out to a more general starting point (somewhat like the [[Source Notes/Tree of Thoughts - Deliberate Problem Solving with Large Language Models]] approach) - Tool failure needs to be discriminated e.g. timeout isn't a failure to solve a problem (e.g. a math problem definitively has no solution) - How are tools not just APIs? - Hypothesis + ai augmented testing of AI apps - AI apps behave probabilistically so you need to test them that way too - How could I plug in a hypothesis into a chain - how to specify which plan step should the hypothesis stub in for? - "Run until evidence for question / claim is satisfied". - need to burn down some value to either track convergence or prune dead ends. - could be mix/max of multiple aspects of a budget e.g. wall time, attempts, tokens, api calls. - but we want to pause state rather than forget state so that human input could allow it to continue (e.g. if we think progress is being made) - Still want to produce an answer in those cases e.g. "Given the current evidence I was able to gather in `<X>` budget, this is inconclusive, here's the evidence, here's the possible alternatives" - Kick off hypotheses from these - We could build up to Causal Intervention Learning e.g. [[Categories for AI talk - Causal Model Abstraction & Grounding via Category Theory - by Taco Cohen]] - Plan / strategy building requires an LLM to be able to say 1) break up into sub-problems (Goals). 2) sub-problem frontier, some of the problems might be of a known kind / have a matching solution, others might not yet. 3) match goals to tools for reaching those goals. 4) how to evaluate if a goal is reached (might require reasoning) 5) we want some measure of result uncertainty - Eval - of each goal's result - of the overall result - of the plan - Eval in the loop => augmented dynamic planning (e.g. how well does this satisfy the goal, continue refining, how the eval failed could be used as input for directions to continue refining) - need a trace of the explanation and provenance of answers along with goal satisfaction - e.g. if I used the google answer block in the serp api result - might also want to track counterfactuals? e.g. serp api had organic searches and they were ignored, considered - might want to log that structured data + embeddings of those states as short term memory that's possibly persist-able. - Agents need to run in an open world. e.g. "I could solve this problem if I had a way to do X", human adds a tool, problem solving continues. - Modal logic inspired responses and hypothetical? - e.g. probably not, almost always - ReAct - reasoning to act ## Timelines - Durative Evolving Goal Frontiers Robotic systems, like [EUROPA 2]([nasa/europa (github.com)](https://github.com/nasa/europa)) can have multiple, sometimes competing, goal frontiers that need to coordinate use of shared or mutually conflicting resources (e.g. single arm end-effector, must stow drill components while in transit but not while rotating camera). ![[_Media/ROS - T-REX multiple timelines.png]] ^[[trex/DivideAndConquer - ROS Wiki](http://wiki.ros.org/trex/DivideAndConquer)] Traditionally, planning was thought to be too expensive to run within the realtime control loop. This lead to centralized plan deliberation and decentralized goal states for all of the control loops. T-REX shows that plan deliberation can be fast enough for many cases and can be decomposed hierarchically. Only global constraints or conflicting sub-goals need to be resolved by a higher level executive. ## Resources - T-REX (teleo-reactive executive) - Hierarchical goal timeline decomposition - [trex/Timelines - ROS Wiki](http://wiki.ros.org/trex/Timelines) - Multi-timeline constraint-based temporal planning [trex/CTP - ROS Wiki](http://wiki.ros.org/trex/CTP) - [trex/SensePlanAct - ROS Wiki](http://wiki.ros.org/trex/SensePlanAct) - [trex/DivideAndConquer - ROS Wiki](http://wiki.ros.org/trex/DivideAndConquer) - [trex/LearnMore - ROS Wiki](http://wiki.ros.org/trex/LearnMore) --- - Links: [[Robotics Automation - Teleo Reactive programs]] [[Plenty/Farm Exec - BT reactor structure|Farm Exec - BT reactor structure]] [[Reactive programing]] [[Stream Processing - Akka streams and actors]] [[Categories for AI talk - Causal Model Abstraction & Grounding via Category Theory - by Taco Cohen]] [[Causal Models]] - Created at: 2023-06-15