high12noon blog

TM4C129ENCPDT, TivaWare, and lwIP in a Polled Superloop

Back to main page

From the Your Mileage May Vary department...

November 11, 2018

I recently worked on a project that runs on the Texas Instruments EK-TM4C129EXL Crypto Connected LaunchPad. This board uses the TM4C129ENCPDT microcontroller, which has an ARM Cortex M4F core, numerous peripherals, a good deal of RAM and Flash memory, and some really nifty features.

It also includes Ethernet.

Many microcontrollers that feature Ethernet connectivity have an integrated MAC (Medium Access Control or Media Access Control) and expose signals that must be connected to an external PHY chip to access the physical network layer. But the nice thing about the TM4C129ENCPDT (and many other part numbers in this microcontroller family) is that it also incorporates the PHY, which saves on board space and reduces the BOM (Bill Of Materials).

This integrated PHY also comes with some nifty features, like including hardware support for IEEE 1588-2002 and IEEE 1588-2008 PTP (Precision Time Protocol) by providing precision timestamping of PTP packets as close to "the wire" as possible, thereby eliminating or reducing the effects of latencies higher up in the network communication stack on the accuracy of timing measurements. PTP is intended for synchronizing clocks across multiple computers in a network with much better accuracy than NTP and with lower cost than GPS (or for applications that cannot use GPS because they are, for example, buried deep underground or whatever). I have not had a chance to play with this PTP very much, but it's on the (very long) TODO list. :-)

But this post is about software, not hardware. It is primarily about how I got network connectivity with lwIP running on the TM4C129ENCPDT in a polled superloop without interfering with some timing-critical interrupts that I have in my firmware.

lwIP, which stands for "lightweight IP" (or, more fully, "lightweight Internet Protocol") is a free software / open source (whichever is your favorite term -- some people prefer one or the other for various reasons) modified-BSD-licensed TCP/IP stack for embedded systems. Originally started by Adam Dunkels many years ago, it is under continuous development, now headed by Simon Goldschmidt and Dirk Ziegelmeier, with about 35 active developers. lwIP is used on a wide variety of embedded platforms, as can be seen from discussions in the active lwIP mailing list and by looking at various embedded platforms that bundle it with their software development libraries.

The lwIP homepage is at: http://savannah.nongnu.org/projects/lwip/.

Now, on TI's TM4C12x microcontroller platform, there are three main routes (that I know of) that you can take to write firmware:

I've used the second method: TivaWare. TivaWare is not an RTOS; rather it is a library of functions to set up and access the various features of the MCU and its peripherals. Your code runs in a single thread (which I call "main context" because it runs your C main() function and its callees), with the possibility to use any number of the many available interrupts to service timing-critical things, such as when a timer fires, data arrives at a UART, a DMA transfer completes, analog samples are available, etc.

There are three main routes that you can take to get your device talking on a network:

As you can guess from what I wrote earlier in this post, I've gone by the second route: lwIP with TivaWare, which implies a single-threaded firmware (with the exception of interrupts that get executed "concurrently" with the main-context code).

Sometimes this type of program is called a "polled superloop" or (if you want to stretch the terminology a bit) a "cooperative multitasking" program, where there is a "while (1)" main loop in the C main() function that executes endlessly and services any number of things in order.

Let's talk about interrupts for a moment. When you use interrupts, you have to take the same care as when designing a multithreaded application (also known as a concurrent application). That is, you must not let one context (a thread, an interrupt, main) step on another context's toes. A really deep discussion of that is beyond the scope of this particular post but there is much information out there on proper concurrent programming, and maybe I'll write a post about it one of these days.

One of the things concurrent applications (or single-threaded applications which use interrupts) must be mindful of is critical sections. Say there's an interrupt that runs every 1 millisecond, collects information, and puts it in a buffer somewhere; and say that main context every so often grabs the information from that buffer and processes it. Because the interrupt can come at any time, including at the very moment that main is accessing that buffer, there has to be some synchronization mechanism in place to make sure that main doesn't read corrupted information, which would happen if main starts to read the information, and in mid-read the interrupt comes along, pauses main without main knowing it, and puts different information there. That would cause main to get half old information and half new information, and chances are it would just be gibberish at that point. The easiest way to prevent this is with critical sections. Just before main goes to look at the buffer, it first acquires the critical section, which can mean that main temporarily disables interrupts, which will prevent the interrupt from firing until main re-enables interrupts. Then main can read the information without fear of the rug being pulled out from under its feet. And finally, main can re-enable the interrupt.

There's one annoying issue with critical sections and that is that when you enter one, you should get your work done as quickly as possible and then get out. If you are in the critical section for a long time (e.g., say main spends 5 milliseconds in the critical section, with interrupts disabled) then you get lousy performance (in this case, the interrupt that should run once every 1 millisecond will fail to run during those 5 milliseconds). So you have to be careful.

Back to the application running on my TM4C129ENCPDT with a network connection...

This application needed to collect information much more frequently than every millisecond. The time between samples was in the fractions of milliseconds. I needed this information collected, processed, built into network packets, and reported to a PC that could do something useful with the information. I also needed a basic web interface served by the TM4C129ENCPDT (not much unlike the way your network router or network connected printer probably has a web interface built in).

I've done embedded networking and embedded web pages before but this was the first time with TI microcontrollers and with lwIP, so in terms of software, this was unfamiliar territory.

So now, finally, I'm getting around to the meat and potatoes of this post. This post documents the approximate steps I took to get the lwIP TCP/IP stack running and serving a web page on a TI EK-TM4C129EXL Crypto Connected LaunchPad.

I talked about how I did not use any RTOS or multithreaded application, but rather used a polled superloop with interrupts doing the timing-critical things.

Let's list some details:

I am using TivaWare version 2.1.4.178, which as far as I know is the newest at this time.

This TivaWare version comes bundled with lwIP, but it is lwIP 1.4.1, which is an older version of lwIP than the newest one available at this time. That's another thing on my endless TODO list: get the newest lwIP (which at the time of this writing is 2.1.1) working on this platform.

Anyway, if you are developing on a Windows machine and installed TivaWare in the default location, then the bundled version of lwIP 1.4.1 is installed under C:\ti\TivaWare_C_Series-2.1.4.178\third_party\lwip-1.4.1.

I began my adventure with the enet_io example. This example program is found (again, if you install to the default location) under C:\ti\TivaWare_C_Series-2.1.4.178\examples\boards\ek-tm4c129exl\enet_io. It connects the board to the network and presents a website, which displays text and images, and through which the user can toggle an onboard LED and perform some other interactive actions.

I imported enet_io into my CCS workspace (CCS is Code Composer Studio -- which is a TI-branded and enhanced version of the Eclipse IDE for TI's microcontrollers and processors). I built and ran that program successfully (as I had fully expected would happen) on my EK-TM4C129EXL.

Having demonstrated that this program works, I returned to my firmware project and added new files, which I called Network.c and Network.h. I call the combination of a ".c" and ".h" file a "module" -- not sure if other developers use that terminology. This "Network" module's purpose is to initialize networking and get it going. I put my network initialization function here, which I wrote by following along in enet_io to see what it does and adding relevant calls to my code, taking care not to conflict with my existing application. I copied several functions, tables, and defines out of enet_io, on which the httpd server depends for CGI, etc., again, taking care to fix issues that would conflict with my existing code. I copied into my project the files io.c, io.h, cgifuncs.c, cgifuncs.h, io_fs.c, and io_fsdata.h from the enet_io example program. These files would ultimately not remain in my actual firmware but were needed to satisfy dependencies as I transitioned from example program to my real program.

The enet_io example configures three interrupts of significance: SysTickIntHandler, lwIPEthernetIntHandler, and AnimTimerIntHandler.

I eliminated AnimTimerIntHandler and rewrote io.c to deal with IOs in a manner that doesn't conflict with my application.

lwIPEthernetIntHandler is the handler for the chip's Ethernet interrupt and is implemented in lwiplib.c, located under TivaWare's utils directory.

This leaves SysTickIntHandler. Because my application already uses the Cortex M4's SysTick for other purposes, I had to modify this a little bit. I configured Timer2A to generate this interrupt instead. I configured the priorities of these interrupts similarly to how it's done in enet_io (e.g., my Timer2A interrupt had a slightly higher priority than my Ethernet interrupt for reasons explained in a comment in enet_io.c).

I added the necessary include directories and include headers to my project.

And then! ... Once dependencies were satisfied and the code built, my program got stuck in IntDefaultHandler(). This is what happens when something is seriously messed up in the code. I wasted all kinds of time trying to trace the cause of that, only to discover that I had forgotten to actually set up the Ethernet interrupt vector in the C startup file! Other MCU platforms handle setup of interrupt vectors differently. For example on Microchip MCUs, there is a name for each interrupt vector, so if you want the interrupt for a certain timer, you write a function with that name and the appropriate GCC attributes and it works. In contrast to this, on TI's TM4C12x platform, you can use any function name you want for the ISR, but you need to tell it which function to use by filling in its name in a table of interrupt vectors located in the C startup file, which also contains code that runs before main() starts. One method isn't necessarily better or worse than another, but I forgot to do that, so when the interrupt fired, code went off into lala land instead of going to my ISR.

The other error was that I forgot to turn off the Timer2A interrupt flag in the Timer2A ISR handler. Doh! This omission (which is a common mistake -- at least one that I commonly make) caused the code to get stuck in that interrupt. As soon as the interrupt exited and main context should have resumed executing, bam! It saw that the Timer2A interrupt flag was still set and so it went right back into the interrupt, over and over again.

Other than these two mistakes, the process went pretty smoothly.

Having done all of this, the program built and ran. It displayed the same webpage as the enet_io example, while the rest of my firmware code appeared to run.

But there was another problem. I found that the web page and lwIP code was interfering with my sub-millisecond, very important, really timing-critical, interrupt. And I couldn't have that because it messed everything up. :-(

Now, lwIP is designed to work with or without a RTOS. When running without a RTOS, the TivaWare port of lwIP does everything in the Ethernet ISR handler. The entire lwIP stack, httpd server, and basically everything related to the network interface, all runs in the Ethernet ISR handler. I knew this when I incorporated the networking stuff into my program, so I set this interrupt to be the lowest priority in the system, so that my more timing-critical interrupts would preempt it. Despite that, it interfered with my more timing-critical interrupts whenever a web browser loaded the webpage served by the board firmware. Now, I did not take any measurements with an oscilloscope or anything, but I knew that something was amiss because I was getting junk data instead of good data out of the board. Obviously, this should never be allowed to happen.

The first thing I investigated was whether IntMasterDisable() is called anywhere in the lwIP code. IntMasterDisable() is the TivaWare function to disable interrupts. Now, lwIP is a system-independent implementation. lwIP is not based on TivaWare or written specifically for TI microcontrollers. Rather, TivaWare incorporated a copy of lwIP with some modifications to interface it to TivaWare -- a "port" of lwIP if you wish to call it that. I did a text search throughout the source code files for IntMasterDisable() and found it in sys_arch.c in the implementation of sys_arch_protect(). This is called through a define, SYS_ARCH_PROTECT(), in various parts of the lwIP code, to protect the stack from the reentrancy issues I described earlier when I talked about concurrent programming and contexts stepping on each others' toes.

A blanket interrupt disable is something I cannot allow, especially when it takes a long time and is unpredictable, like the loading of a webpage, which can happen at any time and is outside my firmware's control.

Therefore I decided to make all of lwIP run in main context (polled in my superloop just like everything else in my program), eliminating this time-lengthy interrupt entirely. And I did it like this:

  1. I copied the lwIP code to a new directory, leaving the original TivaWare-bundled lwIP code untouched (in case I mess it all up). I also copied lwiplib.c and lwiplib.h, which are found in TivaWare's utils directory.
  2. In my copy of lwip-1.4.1/ports/tiva-tm4c129/sys_arch.c, I made sys_arch_protect() and sys_arch_unprotect() empty functions. Since all lwIP code will be called from main context and never from interrupt, and since this is NOT a multithreaded RTOS-type application, there should be no issue of reentrancy.
  3. In my copy of lwip-1.4.1/ports/tiva-tm4c129/netif/tiva-tm4c129.c, in tivaif_hwinit(), I removed the lines "IntEnable(INT_EMAC0);" and "IntMasterEnable();" and left all other initialization as-is.
  4. In my copy of utils/lwiplib.c, in lwIPTimer(), I removed the line "HWREG(NVIC_SW_TRIG) |= INT_EMAC0 - 16;" because this was triggering the Ethernet interrupt via the SWTRIG register. This function now has the effect of only incrementing g_ui32LocalTimer by ui32TimeMS, required for servicing the IP stack's timed handlers. I call lwIPTimer() from my 10 ms Timer2A interrupt. This is the only code I call from interrupt context, and as explained all it does is increment a variable.
  5. I removed lwIPEthernetIntHandler from the vector table. Instead I am now polling it from my superloop -- in other words, each iteration of the loop, I call lwIPEthernetIntHandler() and it services whatever work it needs to do in the networking stack.
  6. Besides the above changes, I changed my project's include paths to use my modified lwIP instead of the original.

The program built and ran. The webpage was served successfully. But did it interfere with my timing-critical interrupts whenever a web page was loaded? In Firefox there is a hotkey combination to force reload a webpage (bypassing the browser's cache). To "stress test" my firmware, I held down this key combination for some time, rapidly reloading the page over and over again, while seeing no interference with my timing-critical interrupts.

Success!

Of course, it will take more tests to see if this setup really works well, not to mention that I'd really like to use the newest lwIP, but so far so good!

Back to main page

Send feedback to: the name of this blog at mail dot com.