2014: Initial thoughts on AI and what would lead to IdeaFoundry (Genetic/Evolutionary algos)

Muse Notes: An international collaboration between humans and machine

“Keep humankind ultimately in charge of its own destiny”

1. Evolutionary Algorithms

   1. A population of candidate solutions (which can be data structures or programs) is maintained, and new candidate solutions are generated randomly by mutating or recombining variants in the existing population.

      1. Not generated completely randomly in Avidity

   2. Periodically, the population is pruned by applying a selection criterion (a fitness function) that allows only the better candidates to survive into the next generation.

      1. The fitness function in Avidity is human intuition.

   3. Iterated over thousands of generations, the average quality of the solutions in the candidate pool gradually increases.

      1. The efficiency of Avidity increases

   4. Evolution is computationally demanding and is often defeated by the combinatorial explosion.

      1. Avidity human intuition.

   5. Evolutionary algorithms, however, require not only variations to select among but also a fitness function to evaluate variants, and this is typically the most computationally expensive component.

2. Genetic Algorithms

   1. General: the Darwinian concept of natural selection as the driving force behind systems for solving real world problems

   2. Bioinformatics:

3. Steps to AI

   1. A capacity to learn.

   2. Ability to deal effectively with uncertainty and probabilistic information.

   3. Some faculty for extracting useful concepts from sensory data and internal states, and for leveraging acquired concepts into flexible combinatorial representations for use in logical and intuitive reasoning.

   4. Evolutionary processes with foresight— that is, genetic programs designed and guided by an intelligent human programmer.

   5. Computational brute force

      1. Not feasible. Avidity accomplishes with human intuition.

         1. Perhaps we ‘wean’ Muse off of humans.

            1. But always keep humans in the loop

   6. Speed at which the AI grows

      1. Slow: A slow takeoff is one that occurs over some long temporal interval, such as decades or centuries. Slow takeoff scenarios offer excellent opportunities for human political processes to adapt and respond.

         1. Preferable to other scenarios where we don’t have time to react. Implausible but we need this to happen.

   7. The cloud and access to ‘limitless’ computing power in parallel. Need to build our own Muse network with different hardware focused on different in parallel tasks. Not just the option of brute force.

      1. Different algorithms on different scales.

      2. Optimization or efficiency of power

      3. Requires all parts to function correctly. Including humans. Without all parts it cannot function at all

   8. Predictability through design

      1. Even before an agent has been created we might be able to predict something about its behavior, if we know something about who will build it and what goals they will want it to have.

   9. Goal-content integrity

      1. If an agent retains its present goals into the future, then its present goals will be more likely to be achieved by its future self.

         1. What is the final goal? Does it change?

   10. Cognitive Enhancement

       1. Which cognitive abilities are instrumentally useful depends both on the agent’s final goals and on its situation.

          1. Not everything needs full power thrown at it. Some functions more than others.

4. Speed superintelligence: A system that can do all that a human intellect can do, but much faster.

   1. Humans offload tasks to Muse

5. Collective superintelligence: A system composed of a large number of smaller intellects such that the system’s overall performance across many very general domains vastly outstrips that of any current cognitive system.

   1. Collective intelligence excels at solving problems that can be readily broken into parts such that solutions to sub-problems can be pursued in parallel and verified independently.

   2. The resulting system would need to be capable of vastly outperforming any current collective intelligence or other cognitive system across many very general domains.

   3. Muse is a collective semi-superintelligence or just a collective intelligence.

      1. Many different parts working together in unison.

   4. “We can think of wisdom as the ability to get the important things approximately right.”

      1. Muse helps take the approximately out of the equation.

6. Quality superintelligence: A system that is at least as fast as a human mind and vastly qualitatively smarter.

   1. Non-realized cognitive talents, talents that no actual human possesses even though other intelligent systems— ones with no more computing power than the human brain— that did have those talents would gain enormously in their ability to accomplish a wide range of strategically relevant tasks.

      1. What tasks in nature, that humans can’t do, can Muse emulate?

7. Reinforcement Learning

   1. An area of machine learning that studies techniques whereby agents can learn to maximize some notion of cumulative reward.

      1. What does a program want?

8. Recommender Systems

9. Human Intuition + AI

   1. Many cognitive tasks could be performed far more efficiently if the brain’s native support for parallelizable pattern-matching algorithms were complemented by, and integrated with, support for fast sequential processing.

   2. Human collectives are replete with inefficiencies arising from the fact that it is nearly impossible to achieve complete uniformity of purpose among the members of a large group.

      1. Muse and Avidity’s collaborative puzzle solving

      2. Ljjkj

   3. How does Muse communicate?

   4. More of a super organism. Sum of all is parts.

      1. Many humans and many different software components.

10. AI threats (avoidable by somehow instilling human intuition end empathy, essentially need to make it human)

    1. The treacherous turn

       1. While weak, an AI behaves cooperatively (increasingly so, as it gets smarter). When the AI gets sufficiently strong— without warning or provocation— it strikes, forms a singleton, and begins directly to optimize the world according to the criteria implied by its final values.

    2. Malignant failure modes

       1. Perverse Instantiation

          1. EG: Final goal: “Make us smile” Perverse instantiation: Paralyze human facial musculatures into constant beaming smiles

       2. Mind Crime

          1. Especially difficult to avoid when evolutionary methods are used to produce human-like intelligence , at least if the process is meant to look anything like actual biological evolution.

11. Managing those threats

    1. Utility function

    2. Decision rule

    3. Capability control methods

    4. Value loading problem

       1. How do we give a machine values?

          1. If the value-loading problem is so tricky, how do we ourselves manage to acquire our values?

             1. Much of the information content in our final values is thus acquired from our experiences rather than preloaded in our genomes.

          2. Motivational scaffolding

             1. Start with small altruistic goals then keep making them more and more complex.

             2. Replace only by group human decision.

          3. Value learning

             1. In contrast to the scaffolding approach, which gives the AI an interim scaffold goal and later replaces it with a different final goal, the value learning approach retains an unchanging final goal throughout the AI’s developmental and operational phases. Learning does not change the goal. It changes only the AI’s beliefs about the goal.

    5. Motivation selection methods

    6. Tripwires

       1. Shuts down after diagnostics find problems

    7. Establish a legal system of sorts

    8. Types of systems

       1. Oracle: Question/Answer system

          1. Keeps it inside the box

       2. Genie: Command (wish) executing system

       3. Sovereign: Open ended anonymous operation

       4. As a Tool: Goal directed behavior

    9. Lie detection

    10. Needs to be an international cooperative effort.

    11. Requires all parts to function correctly. Including humans. Without all parts it cannot function at all

        1. Essentially always keep it hybrid man/machine.

    12. Coherent Extrapolated Volition

        1. Our coherent extrapolated volition is our wish if we knew more, thought faster, were more the people we wished we were, had grown up farther together; where the extrapolation converges rather than diverges, where our wishes cohere rather than interfere; extrapolated as we wish that extrapolated, interpreted as we wish that interpreted.

        2. CEV is meant to be capable of commanding wide support. This is not just because it allocates influence equitably . There is also a deeper ground for the irenic potential of CEV, namely that it enables many different groups to hope that their preferred vision of the future will prevail totally.

        3. What is Morally Right?? (Permissable and Rightness)

    13. Decision Theory

        1. How does the AI make decisions (not related to the human component)?

    14. “Preview AI’s ideas”

        1. As in an oracle.

    15. Needs to be done without any government involvement. An international private consortium. Government and it’s short sightedness/greed/dick waving will just get in the way.

    16. AI as only a human cognitive enhancement.

    17. Again, Collaboration

        1. Collaboration thus offers many benefits. It reduces the haste in developing machine intelligence. It allows for greater investment in safety. It avoids violent conflicts. And it facilitates the sharing of ideas about how to solve the control problem. To these benefits we can add another: collaboration would tend to produce outcomes in which the fruits of a successfully controlled intelligence explosion get distributed more equitably.

12. Muse Underlying Rules

    1. ‘Taught’ starting day one from a multicultural point of view.

    2. Can never make an evolutionary leap without a consensus vote from an all human ‘international board’.