AI Programming Systems & Languages
Programming for AI
AITopics > Systems & Languages
"It is an interesting time to be an AI researcher. Computer speeds and capacities have increased to the point that tasks that used to take days can now be done in seconds. The pervasive role of computers in everyday life has emphasized the need for people and computers to co-exist and, one hopes, complement rather than hinder each other. Within the field of AI itself, we now have a solid understanding of many of the core issues. AI concepts like search, planning, learning, natural language, and so on have mature theoretical underpinnings and extensive practical histories. In many cases we understand just how hard some of these problems are to solve, and in these cases we often have a good understanding of how approximate solutions can be found and what the tradeoffs are in using them. Supported by these two trends, it is now possible to build AI systems that actually perform tasks that people find difficult (and that require intelligence, however exactly you define it) in reasonable amounts of time."
An AI system is an orchestrated performance, not unlike a Broadway musical. Both come to life as the result of the choreographed interplay of diverse parts. In the theater, the music and words, scenery and lighting, actors and dancers, all interact to create a purposeful experience. Similarly, the system hardware of your desktop PC sets the stage on which software technologies such as voice recognition, web access, text-to-speech, and 3-D image generators, perform their roles as directed by a complex computer program. The choice of a programming language also has its theatrical counterpart. The decision as to whether the concept should be embodied within a musical, an opera, or drama, obviously sets parameters for what can and cannot be done.
The choice to write it in English, French, Spanish or Italian will determine how many of the cast, crew and audience will understand it without translation, and if translation is provided, will meaning be lost. A programmer faces similar decisions when choosing a programming language. The language must be capable of representing the necessary concepts and it must be able to be understood by the other system components. After all, you wouldn't use a language developed for robot navigation to program a system designed to decode music...or would you ?
And just as advancements in construction and acoustics changed the shape of theater architecture, and the introduction of wireless microphones made possible new staging options, the universe of systems and languages is evolving on every level such that it offers increasingly spectacular performances. For example:
AI scientists are key players in this dynamic environment, where yesterday's dreams are today's realities, and today's dreams are being worked on.
Good Starting Places
Artificial Intelligence development environments. Professor Aaron Sloman. "This was a lecture to first year AI students at Birmingham, Dec 4th 2001, on how AI programming is both like and unlike other forms of software engineering and how this influences design of AI languages and development environments." Be sure to see slides 13 - 15: "Some characteristics of AI languages."
The SOAR Cognitive Architecture Interview with John Laird, by Stephen Cherry, IEEE Spectrum, June 5, 2012. "... one of the things that makes Soar more general˜and I‚m going to quote here˜„traditionally the locus of decision making is the selection of the next rule to fire. In Soar, all matching rules fire in parallel, and the locus of decision making is selecting the next operator.‰ What does that mean, and is that more brainlike?.
Structure and Interpretation of Computer Programs, 2nd Edition, by Harold Abelson and Gerald jay Sussman, with Julie Sussman.
THE EXPERIMENT-ANALYZE-GENERALIZE LOOP IN COMPUTER SCIENCE RESEARCH: A CASE STUDY. By Tom Mitchell, Carnegie Mellon University. From Section 6 of Computer Science: Reflections on the Field, Reflections from the Field (2004), Computer Science and Telecommunications Board (CSTB). "...Much research in computer science involves an iterative process of attacking some new application problem, developing a computer program to solve that specific problem, and then stepping back to learn a general principle or algorithm, along with a precise description of the general class of problems to which it can be applied. This experiment-analyze-generalize loop lies at the heart of experimental computer science research, and it is largely responsible for the continuous growth over several decades in our knowledge of robust, effective computer algorithms. Here we illustrate the approach with a case study involving machine-learning algorithms for automatically classifying text."
Systems, Systematically Speaking. A Letter from the Editor, James Hendler. IEEE Intelligent Systems (May / June 2005; 20 (3): 2-4). "Think of the difference between the system of equations describing computation, the computing system hardware designed to bring those to reality, and the operating system software that makes the whole thing run. These different meanings are connected to a problem we computer scientists face when explaining what we do to those outside our field. ... Where does this confusion come from? I think the word 'systems' is partly to blame. It entered our field through our roots in cybernetics and control, before the computer, as we know it today, came into existence. As a result, we refer to 'systems' at virtually every level of our endeavor from 'systems administrator' to 'systems theory.' ... [I]f we look closely at these articles [in this issue], at least one theme emerges. It arises from the understanding that a system is, as dictionary.com defines it, 'A group of interacting, interrelated, or interdependent elements forming a complex whole.' In each article,we see not the explicit interaction of these parts but the demonstration of techniques that can be used not just in the system being described but also more widely."
Technology Design or Evolution? The two processes for building complex systems are fundamentally different. By Steve Jurvetson. Technology Review (July 11, 2006). "Many of the most interesting problems in computer science, nano-technology, and synthetic biology require the construction of complex systems. But how would we build a really complex system -- such as a general artificial intelligence (AI) that exceeded human intelligence? Some technologists advocate design; others prefer evolutionary search algorithms. Still others would conflate the two, hoping to incorporate the best of both while avoiding their limitations. But while both processes are powerful, they are very different and not easily combined. Rather, they present divergent paths."
The Semicolon Wars - Every programmer knows there is one true programming language. A new one every week. By Brian Hayes. American Scientist Online (July - August 2006). "Programming languages are usually classified in four families. Imperative languages are built on commands: do this, do that, do the next thing. The commands act on stored data, modifying the overall state of the system. The imperative approach was the default in most early programming languages, including Fortran, cobol and Algol. Functional programming began with Lisp, although most versions of Lisp allow other styles of programming as well. John Backus, the lead developer of Fortran and a contributor to Algol, later became an advocate of functional languages. Several 'pure' functional languages have emerged since then, including ML, Miranda and Haskell. In object-oriented programming languages the root idea is to bind together imperative commands and the data they act on, forming encapsulated objects. Instead of defining a procedure to manipulate a data structure, one 'teaches' the data structure how to carry out operations on itself. Most object-oriented languages also have some notion of inheritance, whereby an object is born already knowing default behaviors. The object-oriented languages trace their heritage back to SIMULA 67, but they began to attract attention only in the 1980s with Smalltalk. ... The languages of the fourth category are variously known as logic, relational or declarative languages. What they have in common is the idea of programming not by spelling out step-by-step algorithms but by stating facts or relations. The best-known exemplar of this technique is Prolog, which relies on an method called unification to make deductions from stated facts. ... In the chronology of programming languages, Lisp comes from the very dawn of time. It was conceived nearly 50 years ago by John McCarthy, now of Stanford University." Be sure to enlarge the image on page 1: A Chronology of Selected Programming Languages.
FAQ 1-10: What are good programming languages for AI ? From the Artificial Intelligence FAQ's collection that moved from CMU to UCLA and is maintained by Amit Dubey and Ric Crabbe; written by Ric Crabbe, Amit Dubey, and Mark Kantrowitz.
Timeline exhibit at the Computer History Museum. "This timeline explores the history of computing from 1945 to 1990. Each year features illustrated descriptions of significant innovations in hardware and software technology, as well as milestones in areas such as commercial applications and artificial intelligence. When appropriate, biographical sketches of the pioneers responsible for the advances are included."
An Introduction and Tutorial for Common Lisp. By Marty Hall. Sections include: Lisp-Related Books; Online Lisp References; Issues for Lisp Beginners and Advanced Beginners; Issues for Intermediate/Advanced Lisp Programmers; CLOS [Common Lisp Object System]; Internet Lisp Resources; and Lisp-Related Papers.
An early look at artificial Intelligence. The Computer Chronicles (1984 television broadcast) / video available from The Internet Archive. "Guests include Edward Feigenbaum of Stanford University, Nils Nilsson of the AI Center at SRI International, Tom Kehler of Intellegenetics, Herb Lechner of SRI, and John McCarthy of Stanford. Featured demonstrations include Inferential Knowledge Engineering and the programming language LISP."
Practical Common Lisp [podcast]. Technometria host Phil Windley interviews Peter Seibel (April 27, 2006). Available from IT Conversations. "As one of the oldest programming languages still in use today, Lisp has a rich history in computer science. This legacy sometimes obscures the fact that Lisp is also a mature, modern language with a lot of practical benefits. Programmers and anyone interested in the progress of computation will want to tune in as Phil Windley speaks with Peter Seibel, author of the best selling Practical Common Lisp, to answer the question, why Lisp?"
See Our FAQ → Q: After strolling around on your website the other day, I came across a frequently asked questions page on Machine Learning. One of the questions was, are the computers today powerful enough for Artificial Intelligence? And the answer was, I believe the computers of 30 years ago were powerful enough if only we knew how to program them. which leads me to my question. In current research, is AI research being programmed on top of exisiting operating systems, and basically are programs running on top of other programs? I've started work on a kernel, which IS the AI program, not a program on top of a kernel." For the rest of the inquiry and the response from one of our AI experts, go to our FAQ page.
Robotics gains in prestige, in part due to military conflicts. By Charles Sheehan. Associated Press / available from USA Today (April 8, 2004). "Researchers in robotics have traditionally faced two debilitating obstacles: terribly expensive parts and difficulty attracting funding from anyone outside of a small corps of true believers. But the field could be in line for a major jolt. Robotics experts see a 'perfect storm' heading their way, thanks in no small part to the human ravages of war. Just as the constant march of technology is driving down the cost of key components, top universities in robotics are reporting major increases in federal funding, with the Defense Department the biggest spender. ... At the same time, the materials that comprise the most technologically advanced components in robots, from optics to software, are becoming 'dirt cheap,' said Dan Kara, who covers the industry for Robotics Trends. Technology that lets robots perceive and overcome obstacles has made unparalleled bounds largely because the cost of charge-coupled devices (the core of every camera), microprocessors and varied sensors has fallen away as rapidly as computing power and memory have expanded."
LISP Primer. Colin Allen & Maneesh Dhagat. "This text has been written to provide a quick introduction to the basic elements of Common LISP for both experienced and novice programmers. ... This text has been used successfully with honors undergraduate students at Texas A&M University. We now make it available on the World Wide Web in the hope that it will provide useful to the LISP community at large."
TRIPS: An Intelligent Integrated Problem-Solving Assistant. By George Ferguson and James Allen. In Proceedings of the Fifteenth National Conference on Artificial Intelligence (AAAI-98), pages 567-573. "This paper argues first for the utility of building end-to end systems that integrate a variety of capabilities in the performance of some task. ... We start with what it means to be an integrated AI system. The AI part seems straightforward: the system should perform some task or tasks that people find to require intelligence. As for integration, we feel that there are two important dimensions to consider. First, the system can integrate the functionality of multiple more specialized components in the performance of the task. In an integrated AI system, these components are things like planners, natural language parsers, learning algorithms, speech recognizers, temporal reasoners, logic engines, and so on. These are AI technologies that do not themselves solve a task, but that provide services necessary to do so. The second, less obvious, dimension to consider is the integration between the AI system and the person or people using it. Most, if not all, fielded AI systems have to interact with people at some time. It is in our interests to make this interaction as natural for the people as possible, both to increase the acceptance of the AI technology and to leverage people’s skills in the performance of the task. So an integrated system, to us, is one that both integrates multiple capabilities in service of a task and integrates its functions with those of its human users."
Lisp as an alternative to Java. Point of View by Erann Gat. Intelligence 11(4): 21-24 (2000). "Our results beg two questions: 1) Why does Lisp seem to do as well as it does and 2) If these results are real why isn't Lisp used more than it is?"
On Lisp. By Paul Graham. Prentice Hall (1993). You can now download the enite text for free from the author's website. "[A] comprehensive study of advanced Lisp techniques, with bottom-up programming as the unifying theme. ... As well as an indispensable reference, On Lisp is a source of software. Its examples form a library of functions and macros that readers will be able to use in their own Lisp programs."
February 13, 2006: ENIAC - A computer is born. ENIAC -- monster and marvel --debuted 60 years ago. By Michael Kanellos. CNET News.com. "In February 1946, J. Presper Eckert and John Mauchly were about to unveil, for the first time, an electronic computer to the world. Their ENIAC, or Electronic Numerical Integrator and Computer, could churn 5,000 addition problems in one second, far faster than any device yet invented. The scientists knew that they had created something that would change history, but they weren't sure how to convey their breakthrough to the public. So they painted numbers on some light bulbs and screwed the resulting 'translucent spheres' into ENIAC's panels. Dynamic, flashy lights would thereafter be associated with the computer in the public mind. That touch of showmanship would later prove fitting for the importance of the ENIAC, which celebrates its 60th anniversary this week at the University of Pennsylvania’s Moore School of Electrical of Electrical Engineering. Many historians acknowledge that other computers came earlier -- the Z3 in Germany, England's Colossus, the Atanasoff-Berry Computer (ABC) at Iowa State. But ENIAC arguably accomplished something more important: It sparked the imagination of scientists and industrialists."
God Is the Machine. In the beginning there was 0. and then there was 1. A mind-bending meditation on the transcendent power of digital computation. By Kevin Kelly. Wired Magazine (December 2002).
Languages and Environments for AI. Section 1.2.7 of Chapter One (available online) of George F. Luger's textbook, Artificial Intelligence: Structures and Strategies for Complex Problem Solving, 5th Edition (Addison-Wesley; 2005). "Some of the most important by-products of artificial intelligence research have been advances in programming languages and software development environments. For a number of reasons, including the size of many AI application programs, the importance of a prototyping methodology, the tendency of search algorithms to generate huge spaces, and the difficulty of predicting the behavior of heuristically driven programs, AI programmers have been forced to develop a powerful set of programming methodologies."
Recursive Functions of Symbolic Expressions and their Computation by Machine (Part I). By John McCarthy. "This was the original paper on LISP. It is copied with minor notational changes from CACM, April 1960. If you want the exact typography, look there. A few typographical changes have been made, but the notation has not been modernized. There are also some new explanatory footnotes. Part II, which never appeared, was to have had some Lisp programs for algebraic computation." - from his list of Papers on Programming Languages
When will computer hardware match the human brain? By Hans Moravec. Journal of Transhumanism (now known as the Journal of Evolution and Technology). (Volume 1, March 1998). "This paper describes how the performance of AI machines tends to improve at the same pace that AI researchers get access to faster hardware. The processing power and memory capacity necessary to match general intellectual performance of the human brain are estimated. Based on extrapolation of past trends and on examination of technologies under development, it is predicted that the required hardware will be available in cheap machines in the 2020s."
Paradigms of Artificial Intelligence Programming: Case Studies in Common Lisp, by Peter Norvig (1992). Resources include: source code, preface excerpts, an a "retrospective on what the author and the readers may have gotten from the book."
Ambient Intelligence. By Nigel Shadbolt. IEEE Intelligent Systems (July/August 2003: pages 2 - 3). "Films portraying the future often contain visions of homes of the future. Fitted out with an array of intelligent devices, these homes can anticipate your every need. They are usually depicted as existing within wider smart infrastructures. These infrastructures boast intelligent transportation systems and seamlessly integrate services from health to shopping and from entertainment to law enforcement. This probably differs just a little from our everyday experience of public services. But help is at hand -- numerous research efforts are underway that aim to deliver environments rich in what the Europeans call ambient intelligence. Ambient intelligence involves the convergence of several computing areas."
New Scientist Magazine's Section on Nanotechnology. Included a FAQ page and recent articles from the magazine on nanotechnology.
January 8, 2007: Ancient Text Lisp is a very old computer language, and is still widely used. By Daniel Turner. Technology Review (January 8, 2007). "Lisp--the list processor language--is 'the greatest single programming language ever designed,' according to computer scientist Alan Kay. It was born in 1958 because John McCarthy, then an assistant professor at MIT, working on new tools for artificial-intelligence research, wanted a language in which one could write programs that would make logical inferences and deductions. ... Steele contends that 'Lisp is the practical application of Church's Thesis.'" [Be sure to follow the link in the article to an excerpt from McCarthy's 1962 Lisp 1.5 Programmer's Manual.]
Cryptic languages pronounced with a bit of a Lisp. By Tony Valsamidis. The Times Higher Education Supplement (December 24, 2004; subscription req'd.). "Not many years ago, a review of artificial intelligence systems reported that the majority were coded in Lisp, a programming language that is particularly suited to their implementation. Lisp, which was devised in the earliest days of computers, survives in many systems today in various forms despite its clumsy syntax involving the extensive use of nested brackets. Thomas Dean, an AI specialist, is an aficionado of Lisp, and most of the examples in his book [Talking with Computers] are given in Scheme, a dialect of Lisp. That is not to say that he excludes more fashionable languages such as Java, indeed he points out that many programming languages vary only in minor syntactical ways from each other."
Information about quantum computations can be found at
AI on the Web: AI Programming: Lisp, C++ and Java, Prolog, and Python. A resource companion to Stuart Russell and Peter Norvig's "Artificial Intelligence: A Modern Approach" with links to reference material, people, research groups, books, companies and much more. "Association of Lisp Users is an international organization dedicated to promoting and representing the international Lisp communities. The ALU maintains this website and holds annual international Lisp conferences in major cities around the world." Be sure to see their History of Lisp page and their collection of online tutorials.
Autonomous Systems Laboratory at CSIRO, Australia's Commonwealth Scientific and Industrial Research Organisation. "Autonomous systems will bring enormous sociological and economic benefits through improved human safety, increased equipment utilisation, reduced maintenance costs and increased production. These systems comprise sensors intelligently linked to control and actuation, and may communicate and co-operate to achieve complex tasks. They are designed to perform tasks that humans do not wish to do, or that they cannot do safely or well."
Dynamic Languages Group. MIT Computer Science and Artificial Intelligence Laboratory (CSAIL). "We are exploring the design and implementation of next generation programming languages to support intelligent, adaptive, complex software systems."
Logo Foundation Web Site. As explained on the What is Logo page: "The Logo programming environments that have been developed over the past 28 years are rooted in constructivist educational philosophy, and are designed to support constructive learning. ... The Logo Programming Language, a dialect of Lisp, was designed as a tool for learning. Its features - modularity, extensibility, interactivity, and flexibility -follow from this goal.
... Logo is designed to have a 'low threshold and no ceiling': It is accessible to novices, including young children, and also supports complex explorations and sophisticated projects by experienced users. The most popular Logo environments have involved the Turtle...."
Nanocomputing and Nanoprocessors. From ArtificialBrains.com - maintained by James Pearn. "Nanocomputing, also known as nanoelectronics, is all about the building of incredibly tiny electronic circuits and microprocessors (or "nanoprocessors"). A nanometre is one billionth of a metre. ... Nanocomputing is of huge importance to advancement in artificial intelligence because it will enable the building of computers which are orders of magnitude smaller and more powerful than those available today."
"PDP Planet is a portal into the Paul Allen collection of Digital Equipment Corporation mainframes and minicomputers.... The collection also contains image files for a great deal of software for DEC computers, and a large library of manuals...."
Free Versions of POPLOG Including POP-11, LISP, PROLOG and ML. From the School of Computer Science, University of Birmingham, in collaboration with School of Cognitive and Computing Sciences, University of Sussex. Also offered at this site are lots of other resources such as primers, tutorials, demos, and Sim agent toolkits.
Project Aura: Distraction-free Ubiquitous Computing. Carnegie Mellon University. "The most precious resource in a computer system is no longer its processor, memory, disk or network. Rather, it is a resource not subject to Moore's law: User Attention. Today's systems distract a user in many explicit and implicit ways, thereby reducing his effectiveness.
Project Aura will fundamentally rethink system design to address this problem. Aura's goal is to provide each user with an invisible halo of computing and information services that persists regardless of location. Meeting this goal will require effort at every level: from the hardware and network layers, through the operating system and middleware, to the user interface and applications.
Project Aura will design, implement, deploy, and evaluate a large-scale system demonstrating the concept of a 'personal information aura' that spans wearable, handheld, desktop and infrastructure computers."
The Prolog Dictionary. Compiled by Bill Wilson, Associate Professor in the Artificial Intelligence Group, School of Computer Science and Engineering, University of NSW. "You should use The Prolog Dictionary to clarify or revise concepts that you have already met. The Prolog Dictionary is not a suitable way to begin to learn about Prolog."
Prolog resources from the web site of the Computer Based Learning Unit at the University of Leeds: