COSA: A New Kind of Programming, Part V

[Repost. See previous post for an update]

Old Links: Part I, II, III, IV, V, VI

Abstract

In Part IV, I described the overall control architecture used in the COSA programming model and what happens to a component when it receives a ‘Start’ or ‘Stop’ signal. In this installment, I explain why simple reactive data connectors are better than message connectors.

The Problem with Messages

Whenever two or more components cooperate on performing a common task, they must not only be attentive to changes in their own immediate environment, but also, to what the other components are doing. When I first devised the COSA model, I had assumed that sending messages was the best way for components to communicate with one another. I have since changed my mind.

The problem with sending messages is that the message sender has no way of knowing when the message should be sent and, as a result, the receiver has no choice but to query the sender for info. This complicates things because it requires the use of a two-way communication mechanism involving two message connectors or a two-line multi-connector. Alternatively, the sender is forced to send messages all the time whether or not the receiver needs it. I now think that messaging should be used strictly in conjunction with a messenger component dedicated to that purpose. I am also slowly coming to the conclusion that messaging should be used only in a distributed environment between components that reside on separate machines. This means that the COSA pages, in particular, the operating system and software composition pages, are in dire need of a revision. I will expand on this in a future post.

Reactive Data Sensing

The communication method that I prefer is reactive data sensing in which components simply watch one another’s actions within a shared data environment. This way, sharers can immediately sense any change in relevant data and react to it if desired. Since it is up to the receiving components to decide whether or not a change is relevant to them, it makes future extensions easier to implement. Sure, you can have restrictions such as which components have permission to effect changes but the component that owns the data should not impose any restriction on when or whether those changes are detected and used by others. Reactive data sensing is a simple matter of creating sensors (comparison operators) and associating them to relevant effectors. Effector-sensor association is done automatically in COSA.

In the figure above, the dotted line means that the + effector is associated with the != sensor. The way it works is that the != comparison operation is performed automatically every time the effector executes its operation. If the assigned change occurs, the sensor emits a signal.

Control Timing

Reactive sensing makes sense within the context of behavior control. The primary reasons for stopping or deactivating a component are that a) its services are either not needed at certain times (deactivation saves cycles) or b) they would conflict with the work of another component (this prevents errors). The main job of a COSA Supervisor is to precisely time the activation and deactivation of various components under its charge so as to ensure smooth cooperation while eliminating conflicts and improving system performance.

Data Connectors

A data connector is just mechanism for sharing data. The data must reside in only one component, the owner or server. The latter has read/write permission on its own data. A client component that is attached to a data owner via a data connector has only read permission by default. It is up to the component designer to specify whether client components can have write permission as well. The figure below illustrates the use of both data and signal connectors. To the right is the color convention that I currently use for the connectors. Note that control connectors are signal connectors.
The data assigned to a data connector can be a single variable, a list of variables, a C++ like data structure or an array. In a design environment, double-clicking on a data connector will open up a box showing the type of data that is assigned to it and the effector and sensors assigned to each data item, if any.

In Part VI, I will introduce a new COSA high-level behavior control mechanism with two complementary commands, ‘Pause’ and ‘Continue’.

Related article:
How to Solve the Parallel Programming Crisis

COSA: A New Kind of Programming, Part IV

[Repost. See previous post for an update]

Old Links: Part I, II, III, IV, V, VI

Abstract

In Part III, I introduced the concept of the control effector, a high-level behavior control mechanism that makes it possible to control a COSA component as if it were a low-level effector. In this post, I describe the overall control architecture used in the COSA programming model and I explain what happens internally to a component under control.

Control Architecture

None of this is chiseled in stone but the way I envisioned it, every high-level component should contain a single master supervisor in charge of one or more slave components. The only components that do not contain a supervisor are low-level components, i.e., components that are composed of cells. It goes without saying that a supervisor is a low-level component. (See Two Types of Components). In the figure below, a component is shown with its supervisor and five slave components. Note that only the control connections are shown for clarity. Normally, the slave components will make data connections with one another and some of the connectors may be visible outside the component. Every one of the slave components may have internal supervisors of their own, if necessary.

Only a supervisor can access the control connector (small red circle) of a slave. The latter cannot access the control connector of its supervisor or that of another slave. The control connector of a supervisor component can only be accessed by an external supervisor component. When a supervisor receives a start or stop signal, it may pass it to the slaves concurrently or in a given sequential order dictated by the design. In an actual development environment, the order in which the components are activated can be shown in slow motion by visually highlighting them in some fashion.

Control Effector

A control effector is a special COSA cell that is used to activate and/or deactivate a low-level component. In a previous illustration (reproduced below) I showed a control effector connected to its 3-input multi-connector. That is the default configuration. This is not a requirement, however. It is up to the designer to decide what to do with the start and stop signals. For example, the component may need to initialize or reset certain variables after starting and before stopping. Or it may do nothing other than outputting a ‘Done’ signal when it receives a ‘Stop’ signal (by routing the ‘Stop’ signal to the ‘Done’ terminal). It also has the option of stopping itself for whatever reason by sending a 'Stop' signal to its control effector.
Deactivation

Stopping a component means to deactivate it so that it can no longer process signals. Deactivating a component is a simple matter of clearing a special activation flag in every cell of the component. This causes the processor to ignore them. A component is fully deactivated only if its control effector receives a 'Stop' signal.

Activation

Activating or starting a component is a little more complicated than just resetting the activation flags. Recall that, unlike conventional programming models, COSA is a change-based or reactive model that uses reactive sensing. That is to say, in a COSA program, a comparison operation (i.e., sensor) is not explicitly executed by the program but is invoked automatically whenever there is a change in a data variable that may potentially affect the comparison (See Effector-Sensor Associations). Being a change detector, a sensor must compare the current state of its assigned variable with its previous state. It fires only when there is a correct change. For example, a non-zero sensor fires only if its variable changes from zero to non-zero. The problem is that, when a component is activated, its sensors have no idea what the previous states were. The solution is to set all sensors to their default states upon activation and invoke them immediately afterwards. This way, when a component is activated, all of its sensors perform their assigned tests or comparisons on their assigned data and fire if necessary. A component is fully activated when its control effector receives a 'Start' signal.

In Part V, I will describe reactive data connectors and explain why they are preferable to active message passing.

Related article:
How to Solve the Parallel Programming Crisis

COSA: A New Kind of Programming, Part III

[Repost. See previous post for an update]

Old links: Part I, II, III, IV, V, VI

Abstract

In Part II, I showed that behavior control in the COSA programming model is based on a simple master/slave mechanism that uses complementary start/stop control commands. In this post, I reveal how the same method can be used to control high-level components as well.

Component As Effector

Controlling a component simply means that the component can be seen, for all intents and purposes, as a low-level COSA effector. Every component will have a special control/effector cell connected to a multi-connector with three connections: two input connections for the start and stop signals and one output connection for the ‘done’ signal that is emitted when the component is finished with its task. The control effector can be used both internally and externally.

Connectors are great because they enforce plug-compatibility and make drag-and-drop software composition possible, but they don’t do much for program comprehension. The reason is that part of a connector’s purpose is to hide information so as not to overwhelm the user with too much complexity. My philosophy is that information should be delivered on an as-needed basis only. That being said, it would be nice if we could summons a special view of a group of a component in order to see exactly how they interact together.

An Example

Let’s say we have a water level component that is used to control a pump that maintains water in a tank at a certain level. Suppose we don’t want the pump to be turned on too often for maintenance or costs reasons. To do that, we can use a generic real-time timer cell to wait, say, 30 minutes between activations. We could incorporate the timer cell directly into the water level component but the latter would no longer be a generic component. A better alternative is to create a separate supervisor component that uses the timer cell to control the activation of the water level component. The figure below shows the two components, as they would normally appear.
In a complex program, the supervisor component would normally be extended to control an indefinite number of slave components. Note that, while the figure shows the components involved, it does not tell us how the water level component is controlled. For that, we need a control view. As seen below, the control view is a simple diagram that depicts the manner in which the water level component is controlled by the supervisor. It displays only the control connections; all other connections, if any, are omitted.
Essentially, the water level component emits a signal when it's done. This signal is used to start the delay timer. At the end of the delay, the timer emits a Done signal, which is used to start the water level component and the cycle begins anew.

In Part IV, I will describe the overall control architecture of a COSA component and explain what happens internally to a component when it receives a start or a stop signal.

Related article:
How to Solve the Parallel Programming Crisis

COSA: A New Kind of Programming, Part II

[Repost. See previous post for an update]

Old links: Part I, II, III, IV, V, VI

Abstract

In Part I of this multi-part article, I wrote that the master/slave approach to elementary behavior control in the COSA programming model should be extended to high-level software construction. My thesis is that this control mechanism is so simple and intuitive that it will revolutionize computer programming by moving it out of the exclusive realm of computer nerds into that of the average computer user. Since COSA is inherently parallel, this will, in effect, solve the parallel programming problem. Below, I go over the elementary principles of control used in COSA.

Effector Control

Most of us are familiar with the controls that come with many of our electrical and electronic appliances. Some may be a little bit fancier than others but almost all come equipped with a minimum set of controls: the on/off (start/stop) buttons. To reuse the metaphor of the previous post, we are the masters and the appliances are the slaves. It turns out that motor command neurons in the brain’s basal ganglia use a similar method to control their target muscles: excitatory (start) and inhibitory (stop) signals. The way it works is that the neuron begins firing as soon as it receives an excitatory signal and stops firing when it receives an inhibitory signal. This is what gave me the original idea for COSA effectors. The addition effector shown below will repeat its operation over and over until it receives a stop command.

A single motor command neuron may receive excitatory and inhibitory signals from hundreds or even thousands of afferent synaptic connections. It goes without saying that the basal ganglia are using some kind of error detection mechanism in order to keep all those incoming control signals from conflicting with one another. COSA effectors, too, use a special mechanism that automatically detects command conflicts. It is applied during application development for debugging purposes and it is based on what I call the principle of motor coordination:

No action can be started if it has already started, or stopped if it is already stopped.

In sum, low-level behavior control in COSA is a very simple thing that even children can grasp. In Part III, I will explain how to control the behavior of high-level COSA components by applying the same principles used with elementary objects.

Related article:
How to Solve the Parallel Programming Crisis

COSA: A New Kind of Programming, Part I

[Repost. See previous post for an update]

Old links: Part I, II, III, IV, V, VI

Abstract

A few exciting ideas related to the on-going evolution of the COSA programming model have been percolating in my mind for quite some time. I wrote a little about them in my recent article, Parallel Computing: Command Hierarchy. These ideas form the basis of a radically new way of looking at software construction that is so intuitive, it promises (or threatens, as the case may be) to reclassify computer programming as a mostly geek-only occupation into something that the average computer user can partake in and enjoy. This multi-part article is an account of the reasoning that led to my current thinking.

Something Is Missing

There is no doubt that the graphical approach to parallel programming can do wonders to productivity and program comprehension. It is true that the use of plug-compatible components in COSA will facilitate drag-and-drop software composition but the simple and intuitive feel that one gets from connecting a sensor to an effector is absent in the realm of high-level components.

Even though looking at a bunch of interconnected COSA components may give one a sense of the various functional parts of a program, the manner in which the parts cooperate to accomplish a task is not obvious. Something is missing.
Masters and Slaves

I realized that the only way to solve the problem is to return to COSA’s roots. The COSA philosophy is that a program is a behaving machine that senses and effects changes in its environment. Of course, there is more to the design of a behaving machine than realizing that programs behave. We, as designers, want the program to behave in a certain way, that is to say, we want control. At the lowest level, we can control the behavior of a program by connecting specific sensors to specific effectors. The applicable metaphor, in this example, is that the sensor is the master or controller and the effector is the slave, i.e., the object that is under control. I rather like the master/slave symbolism because it perfectly illustrates the principle of complementarity. Those of you who have followed my work over the years know that I have always maintained that complementarity is the most important of all the COSA principles because it is the basic underlying principle of every complex organism.

In part II, I will describe how behavior control in COSA works at the elementary level.

Related article:
How to Solve the Parallel Programming Crisis

COSA, the Brain and Parallel Programming

The Future of Computer Programming

My recent series of posts on the brain and universal invariant recognition got me to thinking about Project COSA again. Back in October of last year, I wrote a six-part article titled COSA: A New Kind of Programming. In it, I described a simple master/slave control mechanism that is the main approach to designing parallel COSA applications. One of the things that I should have emphasized at the time is that a COSA application is organized like the brain’s memory, i.e., like a tree. In other words, there is a hierarchy of control. I wrote a brief article about this last year called Parallel Computing: Command Hierarchy but I would like to expand on this very important aspect of COSA application design in an upcoming article. Before I do that, I want to publish the New Kind of Programming series one more time over the next few days because I believe that this is the future of computer programming. Hang in there.

See also:

How to Solve the Parallel Programming Crisis

The Brain: Universal Invariant Recognition, Part VIII

Part I, II, III, IV, V, VI, VII, VIII

Abstract

Previously in this series, I described a biologically plausible neural mechanism for universal invariant object recognition based on discrete signal timing. The proposed solution calls for unsupervised learning using a hierarchical memory architecture organized like a tree. At the bottom level are the building blocks of memory, which are short sequences of up to seven nodes each. Depending on the situation, only relatively few branches in the tree are active at a time (attention mechanism). I am claiming that I got the ideas for this architecture from my interpretation of a symbolic vision written down thousands of years ago by a man named Zechariah (God Remembers). In this post, I will expand on the meaning of the metaphors and explain how probability fits into the model.

Note: Let me reiterate that I am a Christian and I believe that the Bible contains amazing scientific knowledge coded in metaphors. I believe that this knowledge is so revolutionary, it will shake the world in the not too distant future. If this aspect of my work offends your worldview, please do not read my blog as it is not meant for you, sorry.

As One Who Is Awakened out of his Sleep

One of the more interesting metaphors in Zechariah’s vision is that the angel wakes him up and immediately asks him to describe what he sees. Why would Zechariah go to sleep in the vision only to be awakened by the same angel who was talking to him earlier? That it is the same angel most likely signifies that the context of the vision has not changed. But why is Zechariah awakened from his sleep? In my opinion, it is because sleep plays a critical role in memory construction. As recently as last week, the BBC was reporting that new experiments confirmed that sleep is essential to memory consolidation, something that was already rather well known in psychology. Zechariah’s sleep metaphor thus seems appropriate. During waking hours, the brain discovers new sequences, many more than it has time to test and group together. The reason is that, while awake, the brain pays attention only to areas of memory that are essential to whatever tasks it must accomplish. The only time left is sleep time.

The Flying Scroll

My understanding of the vision leads me to believe that there is more to memory management than sleep-time consolidation. Zechariah also wrote about the destruction of bad memory. Established branches that are contradictory (their predictions do not agree) are pruned. This is what I think the metaphor of the flying scroll is about. I’ll have more to say about this in a future article.

Consolidation: Two Types of Branches

How does the brain consolidate sequences into memory? Before we can answer this question, we must first determine the true purpose of memory. In my opinion, memory is primarily a recording mechanism that uses past experience to anticipate future. With this in mind, it makes sense to organize memory sequentially so that it can be easily scanned to determine possible outcomes. But it’s more complicated than that. The sensory space cannot be distilled into independent single-dimensional sequences. There are many interlocking sequences running in parallel. The point I am getting at is that we need two different mechanisms to connect the building blocks of memory into a coherent hole. We need a straight sequential branch that stitches one predecessor sequence to a successor. This type of branch is used for normal anticipatory scanning.

We need another type of branch to join parallel sequences that share a common node. This is what gives us true universal recognition. But sharing a node does not necessarily mean that two or more sequences are related. A relation exists only if both sequences correctly agree on precisely when the shared node fires. During sleep, the sequences are played back and sequences that agree are consolidated into the same group while those that don’t agree are separated.

In my opinion, the two types of links are symbolized by Zechariah’s two olive branches, which are not to be confused with the two olive trees. My current understanding is that the normal sequential branch is mentioned in chapter 3 in conjunction with the stone with seven eyes (a sequence). The other type of branch, the one that links shared-node sequences, is described symbolically in chapter 6, verses 9-15. Note that, in a tree-like architecture such as the one I am proposing, what goes for the lowest level of the hierarchy, also applies to the higher levels. This makes it possible for the brain to reason about higher levels without having to directly access the lowest level. In this light, timing information must be stored in all the branches for quick reference.

Why Seven?

Since sequences have no boundaries, dividing them into short segments provides a quick and easy way to precisely synchronize interlocked sequences without having to travel too far from a shared node. But why use a hierarchy based on seven-node sequences? Why not four or twelve? At this point, I am not sure. I know it explains the seven-item capacity of short-term memory. I also know the hierarchical architecture explains the chunking ability of human memory. I suspect that it has to do with determining probabilities and recognizing objects in an uncertain or incomplete sensory environment. I’ll get back to this in a future article.

Probabilities

In my opinion, the brain has little use for probabilities at the early stages of signal processing. Rather, it is much more likely that the sensory cortex assumes a low probability threshold and retains all signal correlations that overcome it. I believe that probabilities are used only at the output stages of signal processing, where they are directly relevant to planning, reasoning and reinforcement learning. Indeed, Zechariah’s text strongly suggests that imperfections are forgiven during memory consolidation. For examples, in Zechariah 3:3-4, Joshua is given clean garments to replace his dirty clothes and later we read that Zerubbabel brings forth the capstone with shouts of “Grace, grace to it!” It follows that what is important during memory building is not the likelihood that a sequence occurs but whether or not it contradicts another sequence.

Conclusion

I believe that the memory architecture that I tried to describe in this article is the main key to understanding intelligence and adaptive behavior. My purpose in writing this series of posts is to create an Internet paper trail, so to speak. I am not asking anybody to accept my interpretation of Zechariah’s vision or my overall approach to artificial intelligence. As they say, the proof is in the pudding. Eventually I intend to incorporate my entire intelligence hypothesis into an actual learning application such as Animal, my chess-learning program. Stay tuned.