39
$\begingroup$

Most of us learned programming using "textual" programming languages like Basic, C/C++, and Java. I believe it is more natural and efficient for humans to think visually. Visual programming allows developers to write programs by manipulating graphical elements. I guess using visual programming should improve the quality of code and reduce programming bugs. I'm aware of a few visual languages such as App Inventor, Scratch, and LabView.

Why are there no mainstream, general-purpose visual languages for developers? What are the advantages and disadvantages of visual programming?

$\endgroup$
8
  • 8
    $\begingroup$ You are right: people think visually. But images of complex code are impossible to grasp in one glance, so where is the advantage? A good programmer has a visual model of the code in his head no matter what is on the screen. Visual languages are, imho, for people who have not (yet) learned how to abstract from a program's textual representation. That said, I believe that textual code has to look nice, e.g. structures and clear, in order to make it navigatable with the eyes. $\endgroup$
    – Raphael
    Mar 20, 2012 at 15:46
  • $\begingroup$ @Raphael, OK, Imagine this, What if I ask you a textual description of a skyscraper instead of its blue-print? $\endgroup$ Mar 20, 2012 at 15:58
  • 2
    $\begingroup$ Visual languages are certainly employed, at least to some degree, in user interface builders. One can even connect buttons etc to the underlying code implementing the functionality without writing any code (except the underlying code). $\endgroup$ Mar 20, 2012 at 16:01
  • 1
    $\begingroup$ @MohammadAl-Turkistany: That comparison is not very good. First, skyscrapers are built "visually", so it makes sense their plans fit; software is at the end of the day text, so it seems appropriate to use text as model. Secondly, I don't believe anyone wants blueprints of multiple overlapping skyscrapers that break the laws of physics; but that is what software looks like today. $\endgroup$
    – Raphael
    Mar 20, 2012 at 16:11
  • $\begingroup$ @MohammadAl-Turkistany I think this is too broad. Your last paragraph contains 4 questions. This is too much. $\endgroup$
    – uli
    Mar 20, 2012 at 16:17

10 Answers 10

37
$\begingroup$

In general, there is a trade-off in programming language design between ease of use and expressiveness (power). Writing a simple "Hello, world" program in a beginner language, such as Scratch or App Inventor, is generally easier than writing it in a general-purpose programming language such as Java or C++, where you might have a choice of several streams to output to, different character sets, the opportunity to change the syntax, dynamic types, etc.

During the creation of App Inventor (which I was part of), our design philosophy was to make programming simple for the beginner. A trivial example was basing our array indices at 1, rather than 0, even though that makes calculations likely to be performed by advanced programmers slightly more complex.

The main way, however, that visual programming languages tend to be designed for beginners is by eliminating the possibility of syntax errors by making it impossible to create syntactically invalid programs. For example, the block languages don't let you make an rvalue the destination of an assignment statement. This philosophy tends to yield simpler grammars and languages.

When users start building more complex programs in a blocks language, they find that dragging and dropping blocks is slower than typing would be. Would you rather type "a*x^2+b*x+c" or create it with blocks?

Justice can't be given to this topic (at least by me) in a few paragraphs, but some of the main reasons are:

  1. Block languages tend to be designed for beginners so are not as powerful by design.
  2. There is no nice visual way of expressing some complex concepts, such as type systems, that you find in general-purpose programming languages.
  3. Using blocks is unwieldy for complex programs.
$\endgroup$
7
  • 3
    $\begingroup$ Can you say that visual goodies don't scale with user's progress? $\endgroup$
    – Raphael
    Mar 20, 2012 at 20:12
  • $\begingroup$ Nice answer. I like the mention of the design trade-offs. $\endgroup$ Mar 21, 2012 at 0:15
  • 7
    $\begingroup$ @Raphael, I think so. That's on of the reasons for which integrated circuit design went from schematic (which is a graphic language) to HDL and synthesis. I think someone interested in graphic languages should study the usages of schematic and HDL in circuit design and why the switch did occur and why schematic is still preferred in some cases. $\endgroup$ Mar 29, 2012 at 9:47
  • 1
    $\begingroup$ @AProgrammer: Interesting. Sounds like "learn visually, master the abstraction". $\endgroup$
    – Raphael
    Mar 29, 2012 at 9:57
  • $\begingroup$ I think people should try to combine the best of both worlds. So for "ax^2+bx+c" I would type it, but it would appear as blocks (or whatever visual thing), and then I could drag it around or make connections graphically. I think it's all a mater of UI design, for which the best compromise is the most effective use of graphical and keyboard controls, and graphical and textual visualization, and I think we can do better than plain syntax highlighting.. $\endgroup$
    – guillefix
    Jun 8, 2015 at 17:19
22
$\begingroup$

Why are there no mainstream, general-purpose visual languages for developers? What are the advantages and disadvantages of visual programming?

Visual languages tend to broken it three broad categories:

  1. Tools non-programmers to perform basic automation tasks. Think Automator on the Mac.
  2. Learning environments where it's not practical to have lots of typing, or where the structure of the program showing logical flow is important. Think Scratch, Alice, etc.
  3. The problem being addressed is a data-flow problem, and the solution to the problem is well modeled by some sort of data flows between self-contained boxes that mimic the physical world. LabView and Ableton both come to mind.

The advantage to visual programming is that you get a high-level overview of system structure. This leads to the immediate problem that when you get detailed, your spaghetti code really looks like spaghetti. A common component of visual languages is some sort of code block or configuration component for the visual element. The problem is that the programmer needs to make sense of disconnected code blocks that may be interlinked in strange ways.

While there is nothing wrong with visual programming, it may be the case that it's simply not a good approach for most tasks.

$\endgroup$
2
  • 1
    $\begingroup$ Just thought I'd let you know that the author of the other answer thinks yours is a good one. :-) $\endgroup$ Mar 20, 2012 at 22:02
  • 1
    $\begingroup$ when referring to Ableton, do you mean Max/MSP? The two are separate projects developed by different organizations and Max/MSP as well as it's open-source sibling PureData are more complex and expressive than your point 3 gives them credit for IMO. $\endgroup$
    – s-ol
    Jun 10, 2018 at 12:46
11
$\begingroup$

There have been numerous visual programming languages, as the following two bibliographies will show: vlib.org and oregonstate.edu.

IMHO they have failed to gain traction, because whilst they are good for toy examples, they fail to manage the multiple levels of abstraction, representation and granularity that large projects require. You would need to look at a system like AutoDesk Revit (a building information management system used by architects and engineers) to see how complex managing visual information can become.

Rather than looking at general purpose programming, visual programming is most likely to succeed as a configuration tool in specialised domains.

$\endgroup$
9
$\begingroup$

Text is visual.

We use all kinds of visual cues in our code. Every use of whitespace (indents, new lines, blank lines, intraline spacing) is directed towards providing visual cues for the functionality of the code. We use all kinds of different syntax to provide visual information on what code is doing. Our editors color our code to make it stand out.

Mathematics is textual. There is all kinds of notation, but in the end it comes down to basically text. They have been doing for hundreds of years.

My point: the textual representation of code is making use of the visual abilities that humans have. We can probably make better use of it, but not by abandoning text.

$\endgroup$
2
  • 1
    $\begingroup$ Nice observation, but your last subclause is a bold claim. Why do you think that more elaborate visual elements than whitespace and different symbols (and maybe colors) can not help? $\endgroup$
    – Raphael
    Apr 22, 2012 at 11:53
  • 1
    $\begingroup$ @Raphael, I'm not saying we can't make use of more elaborate visual elements, that's why I said "We can probably make better use of it." I'm saying it won't happen by throwing out text. All "visual" programming languages I've seen start with the assumption that text is bad and tries to eliminate it and there they are completely wrong. $\endgroup$ Apr 22, 2012 at 13:57
8
$\begingroup$

[For the PDF version of this reply, the figures or diagrams are interactive and dynamic.]

Net Elements and Annotations: A General-Purpose Visual Programming Language

I use graphics to organize JavaScript™ programs that use the Acrobat®/JavaScript API. Each graphic object represents a Petri Net element (place, transition, input or output) or represents more than one Petri Net element. Each graphic object is actually an annotation of the corresponding net element. However, if every graphic object maps to one and only one net element it may be used to generate the net element. And if a graphic object maps to more than one net element and the mapping is well-defined then it may also be used to generate the net elements. Standard Petri Net elements are represented by certain types of graphics: a circle is a place, a square or rectangle or line is a transition, an arrow from a circle to a square is an input and an arrow from a square to a circle is an output. Furthermore, a net element that is represented by a standard graphics has a set of de-facto standard annotations associated with it.

[The types of annotations in a "Standard Petri Net" are found in the PDF version of this reply.]

Carl Adam Petri described most of these ideas (including the types of annotations in a "Standard Petri Net" in his doctoral dissertation (Petri, 1966). He also applied the net elements and annotations to the description of several logic circuits, such as Figure 6.

Benefits and Challenges

A visual programing language may help a computer programmer develop computer programs (Menzies, 2002).

I have at least three reasons why I find net elements and annotations useful (advantages?).

Firs reason. The process logic can be created one element at a time. This means that a net can be extended by adding elements to the existing net (Petri, 1966). For example, a model of a controller can be divided into internal and external components. The internal component regulates the system. The external component interfaces with the environment by accepting input from the environment. Figure 1 is a Petri Net model of the internal component. It is possible to add a Petri Net model of the external component to the Petri Net model of the internal component by adding the appropriate places and transition (Figure 2).

Figure 1 A Petri Net Model of an Internal Component of a Controller (Halloway, Krogh and Giua, 1997) A Petri Net Model of an Internal Component of a Controller (Halloway, Krogh and Giua, 1997)

Figure 2 A Petri Net Model of an Internal and External Components of a Controller (Halloway, Krogh and Giua, 1997) A Petri Net Model of an Internal and External Components of a Controller (Halloway, Krogh and Giua, 1997)

Second reason. The codes associated with each net element can come from more than one “programming language” (Petri, 1973). They can come from a computer language such as JavaScript, COBOL, ADA and an assembly language. They can come from a Mathematical language such as algebraic symbols. They can come from prose encoded in English, German, French, Greek, Tagalog, Chinese, etc. Thus it may be used as a basis for communication and collaboration throughout the software or system development life cycle; and among different users, developers and stakeholders (Petri, 1973).

Third reason. It is possible to focus on certain graphics objects in the net and to write out the code or logic annotations for the related graphics objects. Consider a Petri Net model of a card game in Figure 3. If the arrow for the input P7  T4 is a standard graphics for an input in a Place/Transition Net and if m_7 is the mark for the place P7 then the logic annotation for updating the mark of the input place is m_7=m_7-1. If s_9^- is the status of the input then the logic annotation for updating the status of the input is s_9^-=((m_7<1) ?false:true).

Figure 3 A Petri Net model of a card game A Petri Net model of a card game

I have at least three reason why I find the application of Petri Nets challenging (disadvantages?)

If there are too many graphics objects then it would be difficult to create or to read the net. The difficulty may be mitigated by taking a subset of the graphics and represent them using one, two or three graphics symbol (Noe, 1973; Petri, 1966). For example, if the Petri Net model of a card game in Figure 3 is considered to have too many graphic objects in the diagram, it is possible to combine some of the graphics and still maintain enough information to map the diagram into a computer program. Consider Figure 4, a Petri Net model of the same game found in Figure 3 with high-level graphics (Chionglo, 2016a).

Figure 4 A Petri Net Model of a Card Game using High-Level Graphics (Chionglo, 2016a) A Petri Net Model of a Card Game using High-Level Graphics (Chionglo, 2016a)

In another example, the external components of the controller in Figure 2 can be combined to create a more concise graphic representation as shown in Figure 5.

Figure 5 A Petri Net Model of a Controller with High-Level Graphics for External Components A Petri Net Model of a Controller with High-Level Graphics for External Components

Finally, a mutually exclusive set of places or a mutually exclusive set of transitions may also be represented using a high-level graphics object (Chionglo, 2015).

Second reason. Even with standard graphics, it can be challenging to draw and position graphics especially if one expects the final diagram to be user- or reader-friendly. Some of the decisions for making a user- or reader-friendly diagram include: the proper layout of graphics objects, the appropriate dimensions of the canvas and shapes, the curvature of arrows, the type of arrow heads, the size and font of text, and the choice of colours for graphics and text.

Third reason. It is easy to create annotations of net elements in an orderly manner because every annotation is directly or indirectly related to a net element. However displaying every annotation along with the graphics of every net element may not be a good idea because there could be too much information presented in the diagram. For example, consider a diagram of a Petri Net model of a logic circuit which includes references to all property and logic annotations (Figure 6). [The original model included a test condition for the status of for every output (figure 31 on page 78 of (Petri, 1966)); the test condition was omitted here because it is equivalent to the original model for the given initial marking. Thus every output has one logic annotation for computing the mark of the output place.]

Figure 6 A Place/Transition Net with annotations – based on figure 31 page 78 of an English translation of Petri’s dissertation (1966) A Place/Transition Net with annotations – based on figure 31 page 78 of an English translation of Petri’s dissertation (1966)

One way to mitigate this challenge would be to identify the types of annotations used in the model, and to define graphics objects that include annotations of these types (Petri, 1966). Thus when a Petri Net diagram is composed of graphics objects from the definitions, the interpretation of these objects should include the “invisible” annotations. Figure 7 should be interpreted as a Standard Petri Net (see Annotations of a Standard Petri Net for the definitions); therefore, the logic annotation may be omitted from the diagram.

Figure 7 A Place/Transition Net – based on figure 31 page 78 of an English translation of Petri’s dissertation (1966) A Place/Transition Net – based on figure 31 page 78 of an English translation of Petri’s dissertation (1966)

Another way to mitigate this challenge would be to use form views of the annotations to complement or supplement the diagram(s) (Chionglo, 2016b; 2014). The views may be further divided into smaller views, and each view can be displayed and hidden.

References

Chionglo, J. F. (2016a). A Reply to “How to design a state flow for a react/redux flashcard game?” at Stack Overflow. Available at https://www.academia.edu/34059934/A_Reply_to_How_to_design_a_state_flow_for_a_react_redux_flashcard_game_at_Stack_Overflow.

Chionglo, J. F. (2016b). Two form views of a Petri Net. Available at http://www.aespen.ca/AEnswers/CAPDissF31P78-form.pdf.

Chionglo, J. F. (2015). Reducing the number of net element graphics in a Petri Net diagram using high-level graphics. Available at http://www.aespen.ca/AEnswers/WjTpY1429533268.

Chionglo, J. F. (2014). Net Elements and Annotations for Computer Programming: Computations and Interactions in PDF. Available at https://www.academia.edu/26906314/Net_Elements_and_Annotations_for_Computer_Programming_Computations_and_Interactions_in_PDF.

Halloway, L. E.; Krogh, B. H. and Giua, A. (1997). A survey of Petri Net methods for controlled discrete event systems [electronic version]. Discrete Event Dynamic Systems: Theory and Applications, Vol. 7. Boston: Kluwer Academic Publishers, pp. 151 – 190.

Menzies, T. (2002). Evaluation issues for visual programming languages. In S. K. Chang (Ed). Handbook of Software Engineering & Knowledge Engineering, Vol. 2 Emerging Technologies. World Scientific Publishing co. Pte. Ltd., pp. 93 – 101.

Noe, J. D. and Nutt, G. J. (1973). “Macro E-Nets for Representation of Parallel Systems”, IEEE Transactions on Computers, vol. C-22, No. 8, Aug. 1973, pp. 718 – 727.

Petri, C. A. (1973). Concepts of Net Theory. In Mathematical Foundations of Computer Science: Proc. of Symposium and Summer School, High Tatras, Sep. 3 – 8, 1973, pages 137 – 146. Math. Inst. of the Slovak Acad. of Sciences, 1973.

Petri, C. A. (1966). Communication with Automota [trans. C.F. Greene, Jr.]. Supplement I to Technical Report RADC-TR-65-377 (Volume I). Griffiss Air Force Base, NY: Rome Air Development Center, Research and Technology Division, Air Force Systems Command, Griffiss Air Force Base. Retrieved Aug. 31, 2011 from http://www.informatik.uni-hamburg.de/TGI/mitarbeiter/profs/petri/doc/Petri-diss-engl.pdf.

$\endgroup$
2
$\begingroup$

Johne Percival Hackworth:

While there is nothing wrong with visual programming, it may be the case that it's simply not a good approach for most tasks.

Perhaps, the visual programming languages to date have just been too immature? The notion that advanced visualizations cannot be applied to software artifacts and that they are left completely up to the 'imagination' of each developer to roll their own might be a false assumption. Raising the level of abstraction in a uniform and automated way seems obvious, so as long as the ability to perform low-level abstractions and high execution performance is not sacrificed. This could ultimately lead to domain expert 'programming,' not much different than the way that spreadsheets automated the task of COBOL programmers to manipulate numbers. The primary difference here is replacing numbers with the manipulation of 'general systems.'

$\endgroup$
2
$\begingroup$

You can look at Programming Without Coding Technology (PWCT)

PWCT is a general purpose visual programming language that enables the development of systems and applications by generating interactive steps instead of writing code.

Here is a good place to start and is open source.

$\endgroup$
1
  • $\begingroup$ For a website about a visual programming language that second link sure is overly textual. Not a single page with screenshots. And the Wikipedia link is broken; the article was deleted for non-notability. $\endgroup$
    – Wildcard
    Jun 21, 2017 at 0:13
2
$\begingroup$

a tricky question. visual or flow-based programming has indeed become more used but it is not widely used compared with all programming languages. significant factors are maintenance and standardization. computer code can be printed on pages. it is not always so clear how to print a visual program.

in contrast to the current top answer I would argue there is definitely no inherent theoretical limitation on visual programming power vs textual languages. in fact visual programming may be seen as easier to maintain someday based on faster human conceptualization of the many layers of abstraction. so there seems to be an unmistakable element of social/cultural inertia/conservatism in its uptake.

the visual editors are probably much more complex in their code, and this is formidable considering that even just text based IDEs can be very complex eg Eclipse, note there are visual-programming oriented plugins into some IDEs such as eciplse eg for GUI building. visual programming for GUI building is fairly widespread now.

it appears to me that visual programming use is increasing in select areas and that long from now it may become more common.

lets not forget that visual programming is inherent to EE chip design for decades where it is not an abstraction and (sub)systems are laid out in 2d designs exactly as intended.

the Lego mindstorms kit, now widespread & over a decade old, has always had development software based on visual programming, now streamlined significantly over many versions.

here is an interesting recent article analyzing the history and prospects and suggesting it may become more common for web-based programming. dynamically laying out controls/widgets on a page is a variant of visual-based programming.

another key/emerging area where it is in widespread use in many tools is BPM, business process modelling.

$\endgroup$
1
$\begingroup$

I guess the reason why these solutions are not so popular, because in most cases they can be un-manageable and become a mess.

Almost all the visual programming tools available out there today are part of larger applications and used for a specific purpose (ex: Blueprint in UE4).

Anyway a new one I came across recently for general programming is Korduene which is really not quite general purpose, as it is more for creating windows applications.

$\endgroup$
2
  • $\begingroup$ Do you have any citations to back up your claim that in most cases visual langauges become messy and unmanageable? $\endgroup$ Oct 14, 2014 at 7:32
  • $\begingroup$ Actually yes I do, for instance spaghetti graph, even with the software I mentioned above, the developer him/her-self had that problem (I've been following the development of that app closely), to backup my point, check out this LINK. $\endgroup$
    – NetInfo
    Oct 14, 2014 at 19:10
0
$\begingroup$

I think @Raphael and others are wrong when they say a program is text. It isn't. It's many things. It's telling the computer what to do, or how to do it. (imperactive, declarative programming) The association of programming with text editing is historic and counterintuitive dogma. Which was created by the limited textual inputs/outputs of early computers. People are not text parsers!

While I think that people are right that a fully visual language (where you do anything visually, by connecting elements via mouse and such) is impractical a for a general purpose language, I think all current languages could be and should be moved to an intermediate level. Where language elements have a visual representation, which is saved in a binary language file. The programmer wouldn't type text like crazy, or live under the spell of lines and indentation.

But inserts elements via the most productive combination of hotkeys/commands/UI elements. And only types strings and other data and identifiers. This would make syntax errors impossible and make languages with safety and correctness in mind (eg:ADA) way more productive. And would also make others safer and less buggy, by making whole classes of common errors impossible. (Such as the ones ADA prevents by being cumbersome)

To some degree things were heading this way. By automatic indentation and syntax coloring. Sadly no-one realized (or cared enough) that it's the core concept of the "human text parser" is what's wrong. The emacs vs vim arguments are funny because both are wrong. They are "solutions" to an artificially created problem.

$\endgroup$
6
  • 1
    $\begingroup$ "People are not text parsers!" What are you doing right now? But, in any case, this answer seems to be mostly your personal opinions about what would be the nicest way to program. Are there any examples of languages or research that you can cite that lift this beyond the level of personal opinion? What advantages do you see in storing source code in a binary format? It would seem to me that this would put you at the mercy of bugs in the development environment -- at least when things are stored as text, I can use a different text editor if my favourite one doesn't work for some reason. $\endgroup$ Oct 30, 2016 at 17:26
  • $\begingroup$ @DavidRicherby Naturally there aren't any examples. I was talking about what could be. The binary format could be tailored to the language. It could bring better performance and data security. " It would seem to me that this would put you at the mercy of bugs in the development environment " That's always the case. It would need to be bug free. Just like many other software. $\endgroup$
    – mzso
    Oct 31, 2016 at 8:40
  • $\begingroup$ If the format is standardized as it should it could have different editors. Thought it would be most useful if the visual part is standardized as well. As for a program to store and retrieve data without fault is not an exotic requirement. Many mature software accomplish this, with bugs few and far between. $\endgroup$
    – mzso
    Oct 31, 2016 at 8:48
  • 1
    $\begingroup$ I asked for "examples or research"; you've said there are no examples. That leaves research. You claim that a binary format would improve performance and security. How? We already have a standardized source format: text. Why use binary? A visual programming language is more complicated and more specialized than a text editor: it seems more likely to be buggy and there are already mature text editors. $\endgroup$ Oct 31, 2016 at 9:21
  • $\begingroup$ @DavidRicherby Text is not a source format. It's a plain dumb text file, which stores text. Of course it would be more complicated, is just a simple thing to edit text, not for programming. It would be faster by avoiding text parsing, interpreting. A text file stores characters, a language file would contain the language elements. (plus data) The visual language wouldn't be more complicated, it would be the same in a different representation. Without the handicap of the text editor. PS: I don't know why or how do you expect examples, research for an idea. $\endgroup$
    – mzso
    Oct 31, 2016 at 11:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.