Your script is my command!
December 22, 2009 1:09 PM   Subscribe

[Software Development filter] I am working on a project that uses a PC (Windows) and a dedicated microprocessor to perform a specific task. I want to create a scripting language that will allow the user to write their own sequences for the tool.

The front-end of the software is being created using MS Visual Basic (Visual Studio) 2008. There is a "processing" back-end being created with the same development package, but using C++ to create a DLL. The DLL uses USB to communicate with a PIC (PIC24FJ256GB110) microprocessor that will control elements of the tool (heater, flow controllers, valves, solenoids, and relays) and gather data from other elements (pressure sensors, temperature sensors, liquid sensors, gas chromatograph).

I want the end user to have the ability to write a script and have the tool process the commands.
The script will have a simple language and limited number of commands.

For example:
10 DELAY XXX ms/sec/min
50 FLOW_CONTROLLER 100 ml/min
60 IF PRESSURE_SENSOR_1 > 100 THEN GOTO [line number]
and so on...

I have searched for and found a couple of "tiny basic" interpreters written in C that look like they could be modified and wedged into service. These interpreters use integers only, I might need floating point capability (pressure value comparisons, etc.) One interpreter requires line numbers the other uses labels for GOTOs and GOSUBs. I don't know if I should use line numbers (requires renumbering at some point) or labels (preprocess a symbol table).

I have a feeling that the end users are not very computer savvy so the scripting would have to be easy to understand and provide adequate error messages.

Do you (or someone you know) have an idea of a better path?
Should I try lex & yacc?
Should I just suck it up and try starting an interpreter from scratch?

Any and all replies are welcome.
AskMe has pulled me out from a jam before, PLEASE do it again!
posted by Drasher to Computers & Internet (27 answers total) 2 users marked this as a favorite
Huh -- i'm just thinking out loud here, so everything I say is just as likely to be crap as it is any good, but...

Instead of making the user learn a scripting language, parsing it, etc etc, can you provide UI elements that sort of string a sequence of events together? In PLC programming (which is all I'm really familiar with that's applicable here), Ladder Logic is used to guide engineers to create what effectively becomes a script. At its simplest, it can effectively be dumbed-down into a some UI elements -- a pump with some events, a flow_controller with other events, a large arrow to make a loop, simple conditionals, etc etc. Drag and drop your pump, assign it to one of the pumps the processor is connected to (#1, #2, ...), pick and event (turn off, ie), etc. Repeat for each step. Then you can take these (as an XML file or something) and auto-generate the code needed to do the actual work.

I used something similar to get some technically-incompetent users to "program" their fish feeding equipment, many moons ago... (using VB6 connected serially to PLCs.... I'm jealous of the USB!)

Good luck!
posted by cgg at 1:24 PM on December 22, 2009

Should I just suck it up and try starting an interpreter from scratch?

Almost without a doubt this is the last thing you want to do. Take a page from the game development world -- there are tons of scripting languages that tie into existing frameworks easily (Python, Lua, etc.).
posted by wrok at 1:30 PM on December 22, 2009

Should I just suck it up and try starting an interpreter from scratch?

Don't do this - it might be fun reinventing the wheel, but your time is probably better spent solving problems that have not been solved hundreds of times before. Writting a solid, usable interpreter is a serious task, you would need to put a lot of effort into it before you reach a point where your code is featureful enough for general use.

I would suggest looking at interpreted languages that can be embedded in your program. Provide a few well thought-out functions, and your users won't even realise they are programming in python (or whatever). The choice of language is up to you - the GNU project picked Guile ( a dialect of Scheme ) for this purpose, but it will probably be too unfamiliar for your users.
posted by Dr Dracator at 1:33 PM on December 22, 2009

posted by GuyZero at 1:36 PM on December 22, 2009

Response by poster: cgg: That idea did not occur to me. I kind of like it. I am not sure how to represent loops and conditions though. Is there another reference that shows looping?

wrok: Yeah, I really don't want to go from scratch if I can help it. But I am also trying to stay away from having to add too much to the system as it is. Right now, VB.NET and VC++ (with more emphasis on the ANSI C and less on the ++) are the only tools being used.

Dr Dracator: I don't know Python and adding it to the system as it stands sounds like it might be a bunch of work.

GuyZero: I did some Forth 20+ years ago... I wouldn't know where to start.
posted by Drasher at 1:41 PM on December 22, 2009

I'm not a big fan of the language but TCL was designed for such things and has a very small footprint.
posted by octothorpe at 1:43 PM on December 22, 2009

First, decide on your target machine architecture.
Where are your scripts running? The host? The PIC?

One trick is to design a tiny decoupled VM. This is something that Lego Mindstorms did in their first version. You could download a program which was a sequence of instructions or you could send an instruction to be executed immediately. This let you run (small) programs directly on the brick or run them piecemeal from the host (with transmission delays). (If I recall - my memory is a little hazy)

Then you can compile scripts into your VM or into an intermediate language that executes commands with the actual logic running on the host.

Any reason that you're wedded to BASIC? Using a more modern compiler compiler than lex/yacc (I used Visual Parse++ which no longer exists), I was able to create a C-like language in few days (it had no looping constructs, so that the halting problem was solved out of the gate).

So if it were me, I would probably design a stack machine VM with instructions along the lines of:

I could see a few others.
So, let's call this a simple stack machine. There is a stack which is both arguments call. All instructions either have no parameters or 1 parameter (push for example, needs an argument: what to push).
So your example code would compile down to something like:
0: Push xxx
5: Wait
6: Push 1
7: Dup
8: SetSwitch
9: Push 3
14: Push 1
19: SetSwitch

(and quite honestly, since you're looking at something like three dozen instructions total, that leaves you with something like 210 instructions to play with if your instruction is a byte, which would let you make special case versions of Push, for example, that pushes between say -50 and 50 and similar for branch). Make all addressing relative so you have position independent code.

Writing an interpreter for this would be very easy on PIC and you write a main loop that reads instructions and executes them. The hardest one I think would be Load, which would invoke a loop to read instructions and buffer them into program memory.
posted by plinth at 1:46 PM on December 22, 2009

IMHO, including an existing language is really your best option -- by calling out to a Python or Lua interpreter you remove the need to include any of that stuff in your software and can keep it all at arms-length in an external library. The problem with doing it yourself is that all of a sudden you have a language/interpreter to maintain. Let that be someone else's problem!

If you have to add a scripting language, you're gonna have to do *something* and I'll bet you $10 bucks, leveraging an existing interpreter is the path of least resistance, least LOC, least development effort. :)


Like Dr Dracator said, it will be easy enough to make (whatever language) transparent to the end user.
posted by wrok at 1:52 PM on December 22, 2009

Unless your interpreter is going to run on the host, I would be leary about using some of the existing scripting engines. For example, Lua touts itself as being small and only takes 153K. For small embedded systems like the PIC listed, there is 16K of RAM and 256K of program space. Lua might run, but barely. I think the tiny VM I outlined could fit in under 4K. You could get similar size in a FORTH system.

If you're not going to run scripts on the target machine (ie, it will always be tethered to a host), I'd would just write a solid object model and use .NET assemblies as the scripting language. That would let the scripters write code in VB or C#.
posted by plinth at 1:55 PM on December 22, 2009

If your goal is to get a simple way for basic users to write simple programs then your simplest answer is VB. If you already have to target VB, why not make it clean, sane and well documented. Then when folks ask you "do you have a scripting tool?" you say "Yes, VB." I'm no fan of VB but you already have this 90% done. The problem then is that your users need a development environment in VB, could this be Visual Basic Express?
posted by bdc34 at 2:01 PM on December 22, 2009

That idea did not occur to me. I kind of like it. I am not sure how to represent loops and conditions though. Is there another reference that shows looping?

There are actual several ways to program PLCs; Ladder Logic was the first that came to mind. All are easily converted between each other, and all end up as assembler at the PLC (which, in essence and at its simplest, is what you're trying to build.) I don't have better examples that wikipedia, however -- it's been a while. But do some research into PLCs; like it's been said upthread -- no sense recreating the wheel, or its spokes.
posted by cgg at 2:04 PM on December 22, 2009

Forth is always a good idea for embedded systems, although it is hard to find folks who understand it well in this day and age. Lua doesn't really scale to the small chips and requires a fairly POSIX environment, although eLua is a possibility. I've embedded pymite in the Canon 5D Mark II. It has the limitation that the Python to bytecode parsing has to happen on the host rather than in the chip. Scheme seems like a good choice, but the embeddable guile requires tons of dependencies like gmp that were too hard to port. Perl was a non-starter since it is so tightly tied to POSIXish systems.

I'm curious to hear what you find. I'm looking for a good general purpose, easily embeddable, limited resource language for several projects that I have. I'm still thinking about going back to Forth.
posted by autopilot at 2:10 PM on December 22, 2009

I think plinth just re-invented Forth there.
posted by GuyZero at 2:29 PM on December 22, 2009

I would check out Lua or an embedded javascript. Writing your own language usually does not end well as the complexity starts to overwhelm your initial simple concept.
posted by chairface at 2:32 PM on December 22, 2009

Oh and per other comments, Forth works because it's so simple and you can actually fit the support structure into a PIC unlike nearly everything else.

Or, per another comment, do something like Mindstorms and license/re-invent LabVIEW and have a GUI builder for the basic control structures.

Depending on the target hardware, LabVIEW may be able to program it directly I think. Buy is always a lot easier than build.
posted by GuyZero at 2:33 PM on December 22, 2009

Response by poster: plinth: Quite an answer! Alas, no, the interpreter will be on the PC and feed commands to the PIC. I looked at references for Visual Parser++. The author site is off the web. I found a downloadable copy, but I'm not sure it it the right thing to do... It is shareware with no one to pay (and no one to get an unlock key from either.)

To those suggesting Python: I am a little leery. I would have to do this embedding AND learn a new language to create the interpreter??!? Yikes!

Sorry if we go off on a PIC tangent.
The idea here is that the user will write the script (in whatever form) on the PC. The PC will then feed instructions to the micro. The firmware on the PIC is slated to be device drivers at best.

My thoughts are leaning towards a "captured" symbolic script with a finite number of statements, maybe selected from a dropdown list. Kind of like the ladder logic idea, but done in such a way that the user does not "write" (type) the code, but selects statements and variables from a graphic environment.

I am still going to look at the Lua idea.
BTW, LabView and I don't really get along well... it cost me my last job.
posted by Drasher at 3:42 PM on December 22, 2009

Definitely, if you're running the script on the host, don't reinvent the wheel. Lua, Python, Iron Python, .NET assembly are all good options.
posted by plinth at 5:17 PM on December 22, 2009

If it helps, what you are trying to create is often referred to as a Domain Specific Language, or DSL. You can use a scripting language like Ruby to create an Internal DSL, which uses the built-in language syntax instead of having to create and parse a new language from scratch.
posted by kenliu at 5:27 PM on December 22, 2009

So - you are creating the host in .NET?

There are literally dozens of ways you could load compiled libraries dynamically and then hook them into an API for your hardware.

Using CodeDOM you could parse a simplified C# or VB.NET, or use C# Script or IronPython or IronRuby or about a dozen other possible .NET dialects and languages.
posted by jkaczor at 5:45 PM on December 22, 2009

I'm with plinth, kenliu, jkaczor and others in saying this is the text book example of a Domain Specific Language. Since you are going to be running this in the .Net platform it will make a ton of sense to reuse one of the existing scripting languages that have been implemented on top of .Net. My personal preference would be IronPython. It was designed from the ground up to be embeddable inside of other .Net applications. With a little effort you can learn Python enough to create the Python objects that match your psuedo code above and that plus a little bit of work to embed IronPython in your app is all it would take to make this work.
posted by mmascolino at 6:35 PM on December 22, 2009

There are black holes in the universe--once entered, they can not be exited.

There are black holes in the universe of design:
'Limited' computer languages.

Take any of the rational suggestions above. You can learn Python much faster than you can recreate even a tiny part of it. And if you don't need Forth, you don't want Forth.
posted by hexatron at 8:20 PM on December 22, 2009

What about an Arduino as the hardware and some kind of simple layer on the top its programming language?
posted by wongcorgi at 9:06 PM on December 22, 2009

Response by poster: I am slightly confused by the resounding Python answer.
While I am not adverse to welding a flavor of Python to the existing project, I don't know enough about it to see why I would use Python over writing it directly in VB or VC.

Can anyone explain?

wongcorgi: While I am close to getting an Arduino for play at home, This project's hardware is locked in stone. The micro board has been developed and the Rev1 board is next to my desk connected by USB and Microchip's ICD3. It is 75% populated with the sensors, etc. I don't stand a chance of convincing the EE (hardware) guy to change that in midstream.
posted by Drasher at 5:53 AM on December 23, 2009

If you're looking to add scripting, Python is well-designed for that. It's easy to extend and it's a well known language. It is fairly performant, etc, etc.

I honestly think that if you're writing to .NET then what you want is an extensibility object model and your clients can code to it in VB, C#, C++/CLI, Boo, etc, etc.
posted by plinth at 7:07 AM on December 23, 2009

What about an Arduino as the hardware and some kind of simple layer on the top its programming language?

Yeah, I was wondering that myself. I remembered programming a fast Arduino script one afternoon (after wiring up a DangerBoard, yay Hacktory) and thought it was a pretty simple and straightforward language.

I'd suggest whipping up a very very simple scripting language if you can. Writing a parser is really not that difficult especially if you keep the language really limited. The thing about python is it's crazy easy to read because it's basically pseudo-code, really, but the whole white-space for blocks thing really will make it a crazy choice for non-programmers.

While I am not adverse to welding a flavor of Python to the existing project, I don't know enough about it to see why I would use Python over writing it directly in VB or VC.

I don't think anyone is telling you to write the thing in Python. I think they're trying to say that the scripting language itself should be Python. Although honestly I'm not clear how it would be able to automagically compile that Python into code that interacted with everything else in the system without making it crazy complicated. Which is why I think something hand-rolled is a better idea. This is especially the case since it is your program that is sending the messages to the controller. I mean, in that case I really, really, really don't see the benefit of Python since the user will either basically be having to write all the control commands themselves (along the lines of ToolController.getTemperature(), ToolController.setFlow(valve,percentage), etc), right? I mean it's not like you can just write
if CoreTemperature <>
and then magically these will actually do something. You'd have to have some other layer in Python handling all of these things, which adds another level of complication.
posted by Deathalicious at 9:34 PM on December 29, 2009

I don't think anyone is telling you to write the thing in Python. I think they're trying to say that the scripting language itself should be Python. Although honestly I'm not clear how it would be able to automagically compile that Python into code that interacted with everything else in the system without making it crazy complicated.

Actually this does turn out to be easy with Python, more specifically, IronPython which is an implementation of Python using the .Net platform. The IronPython interpretor is just a .Net class and it easy to use from any other .Net language. It makes it easy to load and execute a series of textual python commands (either as a string or some sort of text Stream). The interpreter will also allow the python scripts to interact with .Net objects. I would think ideally you would populate the Interpreter object's state with instances of the objects you want your customers to interact with. In which case the code could and would look a whole lot like what Deathalicious has written.
posted by mmascolino at 3:12 PM on January 3, 2010 [1 favorite]

Response by poster: I thank all who have responded to my question.

Unfortunately, I have just received news that the project is about to be cut because of funding.
The customer has requested that we take whatever is completed, put a "face" on it and release the product. (Without any scripting module at all.)

This is very disheartening; however, it is the way of many innovative projects and products in the computer world.
posted by Drasher at 1:43 PM on January 22, 2010

« Older Knob Snob   |   Can we earn money through our online guitar... Newer »
This thread is closed to new comments.