Originally published in the April 1999 issue of:

SCN logo

Serial Killer

by Barry McKinnon

of Mc2Systems Design Group

While the "idea" of computer controlled audio and video equipment seems like it should be the greatest thing since sliced bread, there are days where it's actually about as welcome as a plague rat in your basement.

...it's not just a matter of cleaning the points and wiggling the spark plug wires on the distributor cap

The presence of the increasingly ubiquitous serial port on audio and video gear is the indicator of unprecedented flexibility in the control and application of this equipment... or is it? While the "idea" of computer controlled audio and video equipment seems like it should be the greatest thing since sliced bread, there are days where it's actually about as welcome as a plague rat in your basement. With the advent of computer control in what was a purely hardware based world, we seem to have admitted a few of the less satisfactory aspects of software/hardware interaction into our technical Nirvana.

Everyday as I sit at my computer, working with my everyday office applications, cranking out reports, memos, spreadsheet data, modeling data, etc., I find there are those moments when the software springs into action offering an observation that I appear to be writing a letter, and would I like some help with that? And yet when I actually am looking in every menu and toolbar on the screen, and right clicking on every object for what I expected to be a straightforward formatting command, the same software sits there offering no help whatsoever.

The programmer has made certain assumptions about the grouping of functions and commands that were based on certain assumptions about the users and the use they would find for the software. While that base set of assumptions was likely valid for some narrow set of software flexibility design criteria, the actual use of the software reveals that the idea of a universal flexibility and function model may be flawed. I still find that some task that seems like it should be easy to implement and would be a common requirement is just not possible, or at least not possible in the fashion that I envision.

The computer offers the promise of ultimate flexibility for the applications that run on it, and yet most applications seem to hit limits in their flexibility in completely surprising places. I'm sure I'm not the only person who has thought "Why can't I just grab that on the screen?" or "Why isn't that in the same menu as...?" The reason, of course, is that in order to ever produce a product that can be shipped, the programmers have to reduce infinite possibilities to a more finite set (although from the size of software installations these days, you'd swear they must be approaching an infinite number of lines of code) that can actually be coded and mostly debugged. In that respect computers are very much like sliced bread, as long as you like bread sliced just that thick, and there are enough slices per loaf, they are a wonder (pardon the pun). Now if you happen to want to have some nice thick slices of bread for French Toast, well perhaps sliced bread is not what you want at all.

Now that our industry has made the upgrade from a mass of patch cables and input and output connectors to software controlled widgets where all the signal routing is hidden under the lid and only exposed through the software interface, we find ourselves faced with the same problem as with our desktop computers. When you look at signal flow diagrams of equipment that is computer controlled, you can see the total scope of routing or signal processing flexibility that is available and you can conceive of applications to take advantage of that flexibility. When you actually try to implement that particular application using the software interface, you may find that the programmer hasn't thought about flexibility in the quite the same fashion that you have. Or they have allowed you most of what you want, but by selecting that setting or function, you disable a function that you were hoping to make use of at the same time. And there are those situations where you discover a combination of settings that actually can create entirely new and unforeseen problems, like having input gates and auto-level control fight each other to the point where you can ensure that a microphone will only ever come on once, and then never again.

In the days of a pile of boxes (am I sounding a bit nostalgic?) and a pile of patch cables, you could usually signal trace the problem and discover the cause of unusual signal path glitches, and often remedy it since you have access to all the control ranges of all the devices in hardware. In the land of software control, the best you can do is report the bug to the manufacturer who has to replicate the problem and then get their programmers to see if they can remedy the problem easily, or if it may require a substantial code rewrite, which means waiting until the next software release, which also means having to develop a workaround for the interim, or abandoning the desired feature because the revision won't be available in any useful time frame.

As with desktop software, the hardware manufacturers' programmers have to make choices out of the infinite flexibility that the hardware may offer, and collapse it into a simple and reliable user interface. The simple graphic user interface means you can quickly get at any application that is supported in the interface, but anything unusual may be out of reach. It's like opening a hood on a new 1998 Chevy truck versus your old 1968 Chevy pickup, all of sudden you realize that it's not just a matter of cleaning the points and wiggling the spark plug wires on the distributor cap, you have no idea of what is actually going on in there and it is just as likely a software problem, and not a hardware problem, or possibly a software talking to hardware problem.

Because the graphical user interface limits the amount of flexibility to a manageable set of options, the amount of documentation that is usually provided with the hardware/software is limited to using the interface software to do the setup of the equipment. The documentation is usually intended for a static setup of presets and routing using the software provided by the equipment manufacturer. Although it is possible to control these devices dynamically using the serial port from a control system or other computer interface, the documentation in those areas is often minimal or not thoroughly developed.

The expectation should be that if you want to implement the flexibility or capability beyond that shown in the user interface, then the documentation should be provided to do that through the serial port. If the unit is a matrix mixer, then through the serial port it would be nice to have access to individual crosspoints and not have to deal with calling presets or macros. It would be nice to have access right down to the operation of things like logic switches and LED drivers once you decide to control the system via the serial port. This should be similar to the way computer games grab control of the hardware on a computer, they treat it as a platform where everything is fair game to control.

Serial Killer Poster

...the manufacturer is off-loading a great deal of research and troubleshooting onto the contractor, and as a result the contractor can watch their profit margin dwindle...

...implement individual solutions and workarounds rather than addressing what appears to be a systemic problem in the design of the hardware or software.

This is the area that creates the most frustration for consultants and contractors alike. It frustrates the consultant because it is often difficult, or perhaps impossible, to implement the system features exactly as they might want to, because of decisions made at the programming stage. It frustrates the contractor trying to implement these functions because the manufacturer is off-loading a great deal of research and troubleshooting onto the contractor, and as a result the contractor can watch their profit margin dwindle as they try iteration after iteration of control programming in an effort to establish the required functions. It can be something as seemingly trivial as command pulse length, where the control system command string is held for 50msec and the equipment being controlled wants to see a 100msec long pulse before it will react, or vice versa. When the contractor reviews their control system programming with the hardware manufacturers' tech support, all the codes are correct, it's just a timing issue, a timing issue that isn't actually documented anywhere.

As one contractor put it, much of this equipment would get a note on its report card that said; "Does not play well with others." The whole issue of serial communication between control and controlled hardware is a sore point. While RS-232 serial communication is an extremely flexible protocol, the implementation by manufacturers tends towards excessive flexibility. Or to look at it another way, serial command protocol lacks consistency in its implementation. And that problem exists at several levels, from one model to the next in a manufacturer's product line (or at its worst, from one serial number sequence to the next in the same product model), to completely different implementations by different manufacturers.

This seems to be yet another systematic glitch we have imported straight out of the computer operating system environment. I'm sure we've all had the experience of trying to troubleshoot our computer when either new hardware or software is installed only to find that the tech support personnel will say "Oh, you can't use that video card with our software, you need to have one on this list." or "Oh, you can't use that software with our video card, you need to contact that software vendor for the modified drivers". I'm sorry, but this is pathetic enough in computers, where you would have some basic assumption that there is some kind of operating system/platform standard that would be applicable. This kind of approach in hardware intended for the systems contracting industry is unwelcome to say the least. This kind of random element in systems installation and integration kills the ability of the contractor to accurately predict the required labor for installation, and that is a critical factor in keeping a job profitable. If this was an infrequent problem it might not be a big issue, but with more and more hardware showing up with computer operation or setup capability (or requirements), this is becoming an issue with every installation more elaborate than a paging amplifier and a couple of speakers.

This is no gross exaggeration, I think virtually every one of our system installations currently under way has had some control interface issue that the contractors have struggled with, and those have ranged from minor glitches to a serious hurdle in system function. In many of these situations both manufacturers of control and controlled equipment have provided programming information and reviewed the programming info being used, and the consensus is that it should work. Unfortunately consensus does not actually make the equipment function, and it's the contractor that is caught in the middle. The facility owner does not care if manufacturer A and manufacturer B are not providing the right programming information, they only know that the video projector can't be controlled or the slide-to-video converter isn't yet operational. Both manufacturers will say that it is possible to control the device in question, but it seems no one manufacturer is capable of, or responsible for, actually making that happen. That just leaves the contractor who is caught in the middle of a half-finished system with no good prospect for making further progress.

The manufacturers of audio and video hardware that are purporting to allow serial control of their equipment from either a computer or a control system serial port need to be paying more attention to the importance of this protocol and to they way it is used in the real world (not the programmer's world). If it can be controlled with serial protocol, then it is critically important that the equipment manufacturers disseminate the control protocol information to the major manufacturers of control system hardware. It has been especially frustrating to deal with products where the manufacturers seem to know and understand that serial command protocols change from model to model, or from one serial number group to the next, and yet the manufacturer making those somewhat random changes is not following through to update the control system suppliers on those changes. Some equipment has a reputation for being cantankerous to control and yet there seems to be no effort made to make changes to the hardware or software that might just fix the problem.

There appears to be a large number of industry people who have individually solved these puzzles and problems, and yet these people are often the contractors, who are not the ones that SHOULD have to be solving these problems. Because of this situation, this useful information is not getting back to the manufacturers and the problems continue to propagate as a result. This is the same bad attitude that computer software and hardware manufacturers have adopted; implement individual solutions and workarounds rather than addressing what appears to be a systemic problem in the design of the hardware or software. It is no more acceptable to have the contractor solve the control interface problems than to have a computer hardware or software manufacturer expect the end user to pay for tech support for problems generated by inadequate design forethought.

The equipment manufacturers may complain that it is too expensive to provide this documentation, or to take the initiative to contact the control manufacturers and keep them up-to-date. The reality is that for serial communication to be effective, there first has to be adequate human communication to allow all the machine communication to take place effectively.

Return to the Systems Contractor News Article Index

Return to Barry McKinnon's bio

Return to the Mc2Systems Design Group main page

United Entertainment Media Inc.
460 Park Avenue South, 9th Floor
New York, NY, 10016
Ph. 212-378-0400
FAX 212-378-2160
by e-mail