Learn Who You Can Trust to Provide Accurate Information
Gathering accurate information is an important aspect of troubleshooting. However well meaning, some of the people you deal with will not provide accurate information.
This may seem like stating the obvious, but always try to verify all the information you are given.

Never Compromise The Safety Of The Machine Or The Workers
Many times when you are charged with troubleshooting a system, you are placed in a position of authority. Be aware that others will look to you to provide direction. Make sure that as you provide that direction, you are giving their personal safety your highest priority.

Listen to Your Boss
If he tells you to call a vendor, don’t fiddle around with trying to fix it yourself – call the vendor.

Don’t Let Your Ego Get in the Way
Your job is to fix the equipment. If it means making a phone call to someone you dislike, admitting that you made a mistake in your initial diagnosis, just do it and get over it. In the long run, it is the easy way.

Communication
Keep the lines of communication open. Network with people who may be valuable to you when you’re trying to troubleshoot a system. Keep those who have an interest in the system well informed of problems you discover or changes you make to the system.

It Is Not Necessarily About Knowing One PLC Or Another – It Is About Understanding The Troubleshooting Method
All PLCs are different. All systems are different. The real key to being a good troubleshooter is to understand the Troubleshooting Method.

First, try to understand how to troubleshoot anything and let your method be refined as you gain experience.

DO NOT Compromise The Safety Of A Machine By Altering The Program
Before you make any changes to the program, always ask yourself if this change will affect the safety of the machine or the people who work around it.

No One Expects You To Wave A Magic Wand
No one expects you to wave a magic wand and get the machine to work. What is expected, however, is that you have a goal and a plan, and that you are executing the plan. It is true that, many times, there is pressure in a troubleshooting scenario.

Don’t be nervous; present a blend of humility and confidence. In other words, respect what information is given to you, but be prepared to make your own decisions regarding how to proceed with the troubleshooting.

Know When To Stop
Don’t go past the mark you aimed for; know when to stop.

That moment of victory is often the moment of greatest peril. When you have solved the initial problem, and you are the hero, arrogance and overconfidence can push you past the goal you had aimed for. By going too far, you make new problems for yourself.  Don’t allow success to go to your head.  Remember that there is no substitute for strategy and careful planning. Set a goal, and when you reach it, stop.

This is a trap that many programmers fall into.

Always Know How To Return A Program To Its Original State.

I like to keep a handwritten log of the changes I make anytime I modify an existing program. That way, if things don’t work out, I can simply retrace my steps and make it just like it was.

Additionally, the program that is running in the PLC should be uploaded from the PLC to the laptop and stored in a safe place; perhaps on a thumb drive that can physically be removed from the computer.

However, downloading a program from your laptop to the PLC to restore the PLC to its original condition is not always ideal. First, you have to make sure you have a fresh version of the program. A program that was uploaded and saved a month ago might have old data. You might inadvertently erase any data that had been collected in the last month.

Also, downloading requires you to take the PLC out of “Run” mode and put it in “Program” mode. That means the PLC cannot run the equipment during this time. That might be a problem.

When modifying programs on the factory floor, I believe the best method is to simply write down the changes I have made. In most cases, it is much easier to go back and manually undo the mods you have made, as opposed to downloading a copy of the original program.

It is true that some program modifications are relatively simple. Picture this, though; as you are going through the rungs and changing the logic, you get a call regarding another machine that has gone down. Suddenly, your priorities have shifted. You may have to disconnect from this PLC and take your laptop over to the other machine.

When you return to your original task a few hours later, you might not remember where you left off. Simply referring to your notes is the easiest way to get back on track.

Besides, when the job is done and someone asks you what you did, you can easily pull out your notes and show them.

Sometimes, it is simply about being able to see the big picture.

This is a story I think all of us can relate to.

Let’s take a very simple example. Your wife calls you from the living room and tells you that the light bulb in her favorite lamp has burned out and she wants you to replace it. Being a seasoned troubleshooter, you immediately visualize a clear goal and develop a concise plan.

Even though she said she wants you to replace the bulb, the real goal is to get the light to work. Simply replacing the bulb may not achieve the goal.

In the interest of domestic harmony, there is really no reason to discuss that part with her. You are using discretion, but also realizing that the light bulb may actually be fine, and that the real problem may lie elsewhere.

You might be thinking right now that I am stating the obvious, and in this case, that may be true. On a factory floor, however, be aware that people sometimes develop a plan for you that might not be the best approach.

On your way to the living room, you are thinking of possible causes for the malfunction.

-    Is the lamp plugged in?

-    Is the switch turned to the correct position?

-    Is the bulb secure in the lamp socket?

-    Is there a wall switch that switches power to the outlet that the lamp uses?

-    Could the switch in the lamp be broken?

-    When was that bulb replaced?

-    Do you have a new bulb of the same wattage?

Your first step it to verify that the lamp is plugged in. Given that it is, you operate the switch, yet the bulb still does not illuminate. You reach into the lamp to verify that the bulb is secure and find that it is tight in the socket. Looking across the room, you see a wall switch and remember that this is a switched outlet. Turning on the switch now causes the bulb to be illuminated.

Your wife is happy and domestic bliss is maintained, even though her stated goal of having you replace the bulb was not met (let’s not make a big deal out of that, though).

Suppose, instead of using good troubleshooting techniques, you simply followed her instructions.

You open your utility room door and dig around until you find a package of light bulbs. You grab one and walk out to the living room. As you unscrew the bulb from the lamp, you realize that it is a 75-watt bulb, and that the bulb you brought from the utility room is a 40-watt bulb. You replace it anyway, only to find that it still does not work.

Your wife is still not happy, even though you did what she asked you to do. Domestic harmony is threatened as patience is wearing thin.

Finally, you realize that the real goal is to get the light to work.

As a last resort, you begrudgingly follow proper troubleshooting techniques. You discover that the wall switch had been mistakenly turned off. The lamp now works, but since you already replaced the original 75-watt bulb with a 40-watt bulb, the light is insufficient and your wife is still not happy.

You turn off the lamp, and wait until the bulb is cool enough to change without burning your fingers. Finally, you are able to change out the bulb and your wife is reasonably happy.

You, though, feel a little frustrated in realizing that you spent 5 minutes to fix a problem that should have taken 30 seconds. You show the greatest wisdom, however, by not questioning your wife’s ability to operate the wall switch that would have powered the lamp in the first place . . .

This is a very simple illustration, but it is important that you understand the importance of having a clear goal and a concise plan.

Let’s consider a batching system just like the one we worked on earlier, except that the PLC is an Allen-Bradley ControlLogix. The software we use to program a ControlLogix PLC is Rockwell’s RSLogix 5000.

TROUBLESHOOTING OPERATION #3

BATCHING SYSTEM WON’T PUMP FINISHED PRODUCT

The problem is that the system is not working. A completed batch was being pumped to the filing lines when the pumped stopped and the “System Fault” pilot light came on. Pressing the “Start” or “Stop” buttons has no effect.

Shown below is a graphical layout of the system. In fact, apart from the program, this is all the information you have.

The operator station looks like this.

The “System Fault” pilot light is on.

As you go online, this screen appears.

We want to find the rung that generates a system fault.

In the Controller Organizer (the window on the left), click on the “+” folder icon to open the “Tasks” folder.

Open the “Main Task” folder.

Open the “Main Routine”.

The screen looks like this.

Finding The Problem

We could use CTRL-F to find the system fault bit, but it conveniently appears in the first rung.

The instruction is false, which keeps the output “SystemEnable” false.

Right-click on the tag name (SystemFault) and choose “Find All SystemFault”. The search results box appears at the bottom of the screen.

We are looking for the output, or OTE. Click on the blue line that says “Found: Rung 22, OTE Operand 0: OTE(SystemFault).

We are taken to Rung 22.

The output is true, but there is no other instruction in the rung that is highlighted.

Take a closer look at the GRT instruction. A GRT instruction is true when Source A is greater than Source B.

In our GRT instruction, Source A is 100 and Source B is 95. That makes the instruction true, which is turning on the “SystemFault” output, which is keeping the system from running.

Not all instructions in RSLogix change color when they are true.

According to the description applied to the instruction, Source A is the liquid level in the mixing tank. It is being reported as 100 (whatever that means).

Source A gets its value from Local:2:I.Ch1Data. We can tell that this tag represents a hardwired input.

We can deduce that this is an analog input, since we are getting a value that is not just 1 or 0.

Investigating An Analog Input
In the Controller Organizer, double-click on “I/O Configuration”. On this PLC, slot 2 has a 1756-IF8 card. Right-click on slot 2 and choose “Properties”. The “Module Properties: Local:2 (1756-IF8 1.1)” window appears. Click on the “Configuration” tab. Click on the “Channel 1″ button.

The configuration is shown like this.

Now we see that the input range on this channel is 0 to 10 volts. The engineering units applied to the scaling suggests that the level in the tank is shown as a percentage. That is, 0 volts equals 0% and 10 volts equals 100%.

Let’s go back to Rung 22.

The GRT will become true if the level in the tank exceeds 95%, which sets a system fault. That makes sense.

The PLC thinks the tank is 100% full. As we look inside the tank, we can see that it is nearly empty.

We can conclude that the PLC program is functioning properly. It is doing what it is supposed to do.

This is very likely to be a hardware problem. There could be a short in the signal wire from the level sensor, the sensor could be defective or the input card on the PLC could be bad.

Your electrician measures the voltage on the input of the PLC card and sees 10 volts. He disconnects the signal wire from the PLC, and you see the value of Source A in the GRT instruction drop to zero.

He measures the voltage at the output of the level sensor and again finds 10 volts.

You can manually pump the remaining liquid out of the tank and monitor the voltage at the output of the sensor. If it does not change, then the next step would be to clean, repair or replace the sensor.

The sensor is replaced and you see a more accurate level indicated in the GRT.

Testing the sensor to make sure it shuts down the system if the level gets too high might be difficult without disrupting a batch. You may have to use your imagination here. I will leave that up to you.