"Debug your robot from anywhere" or "ESP-01 as remote flasher"

ESP-01 with ESP-LINK 3 as remote Wi-Fi flasher (OTA) and Selfmade Arduino clone with 20MHz freq.

I want to flash my ATMega (Arduino) over-the-air from Arduino IDE.

I have custom board with ATMega328P, 20MHz oscillator and ESP-01.

This article contains examples only for Windows, because Linux users have good examples and tools in native packages and documentation.

1. Connect ESP module to USB-UART

First of all we need ESP8266 board, I try cheap ESP-01 with 8Mb (1MB) and USB-UART, I use CH340. USB-UART must work on 3.3V!

Usually USB-UART have very low-current 3.3V power out. Then we need 3.3V power supply. We need 3.3V voltage-regulator, may use LM1117-33, 0.5A is enough.

ESP-01 pins

Connect ESP to UART

If you don't have RTS and DTR you can try modify you UART module or continue with manual reset switch.

Just connect GPIO0 to GND and RST(EXT_RSBT) to switch connected to GND.
Or make it like in this serious scheme with 10k resistors :-)

Then power on ESP. This should lit up the red LED and might start blinking the blue LED, which depends on your current firmware.
Let's try to flash.

2. Prepare to flash ESP-LINK

ESP-LINK firmware.

We can flash it with ESP8266Flasher tool.
Just download and extract archive with firmware http://s3.voneicken.com/esp-link/esp-link-v3.0.14-g963ffbb.tgz

Download tool, run it, set COM port (your USB-UART) and four files and addresses to Flash:

In my case we must use these adresses. It depends on ESP8266 board memory size.

See https://github.com/jeelabs/esp-link/blob/master/FLASHING.md

32Mbit / 4Mbyte module
0x00000 boot_v1.5.bin
0x1000 user1.bin
0x3FC000 esp_init_data_default.bin
0x3FE000 blank.bin

4Mbit / 512Kbyte module
0x00000 boot_v1.5.bin
0x1000 user1.bin
0x7C000 esp_init_data_default.bin
0x7E000 blank.bin

For 8Mbit / 1MByte module
0x00000 boot_v1.5.bin
0x1000 user1.bin
0xFC000 esp_init_data_default.bin
0xFE000 blank.bin

Push Flash button, push reset switch if you use manual reset.

Sometimes ESP or UART may stuck and you may try shortly connect reset to GND even if you use RTS for reset.

After successfull flashing try to find Wi-Fi network, smth like ESP_0E5E6E.

You need connect to this network and then go to the http://192.168.4.1/

Then you may connect ESP-LINK to your Wi-Fi net or still use ESP as Access point.

3. Selfmade Arduino bootloader

If you have branded Arduino and want to use stock 16MHz you may skip this part.

I have worked Atmel ATMega328P, nevermind on a breadboard, PCB or smth another.
Ofcourse it maybe branded Arduino.

Main is you must check that it work properly by USB-UART.

I have ATMega328P with 20MHz oscillator and a little problem is a bootloader for 20MHz. Default Arduino's bootloaders are for 16MHz or 8MHz.

If you, like me, want 20MHz you need also ISP programmer for load bootloader. I use USBasp.
This article dont cover programming by ISP.

We need go by cmd shell to
c:\Program Files (x86)\Arduino\hardware\arduino\avr\bootloaders\optiboot\
and compile our bootloader.

Check that you have make utility:
make -v

Check path if you have it:
where make

I can't find "make" utility in Arduino package and I download and install WinAVR package.
Installer may add bin dir to OS PATH and in Windows we must reboot after install WinAVR.

When I try to compile bootloader with arduino's package avr-gcc I recive error while uploading bootloader by ISP. (It have 1581 bytes size.)

I'm not a Programmer and unfortunately I don't know why it don't work.

You can try this for compiling this buttloader
make clean
make OS=windows ENV=arduino atmega328

Actually this two options (OS=windows ENV=arduino) just says to "make" where we have avr-gcc.

But!
When I try WinAVR avr-gcc I recive worked bottloader.

First of all we must edit Makefile

Put this config after atmega328 section:

atmega328_20: TARGET = atmega328_20
atmega328_20: MCU_TARGET = atmega328p
atmega328_20: CFLAGS += '-DLED_START_FLASHES=3' '-DBAUD_RATE=115200'
atmega328_20: AVR_FREQ = 20000000L
atmega328_20: LDSECTIONS  = -Wl,--section-start=.text=0x7e00 -Wl,--section-start=.version=0x7ffe
atmega328_20: $(PROGRAM)_atmega328_20.hex
atmega328_20: $(PROGRAM)_atmega328_20.lst


(optional) You may change flashing speed
atmega328_20: CFLAGS += '-DLED_START_FLASHES=3' '-DBAUD_RATE=115200'

I test 250000, it worked fine on 20MHz, but I recommend use 115200 or 57600 if you have breadboard or something soldering shit-board

(optional) You may also change bootloader Watchdog timer to 2s or 4s. Edit optiboot.c
find and change
// Set up watchdog to trigger after 500ms
  watchdogConfig(WATCHDOG_4S);

Also you may need remove -mshort-calls from Makefile
Find string
OPTIMIZE = -Os -fno-inline-small-functions -fno-split-wide-types -mshort-calls

This option deprecated from new avr-gcc.

Then go to our cmd shell and compile it

c:\Program Files (x86)\Arduino\hardware\arduino\avr\bootloaders\optiboot\>

make atmega328_20

We can receive smth like this:

avr-gcc -g -Wall -Os -fno-inline-small-functions -fno-split-wide-types -mmcu=atmega328p -DF_CPU=20000000L  '-DLED_START_FLASHES=3' '-DBAUD_RATE=115200'   -c -o optiboot.o optiboot.c
avr-gcc -g -Wall -Os -fno-inline-small-functions -fno-split-wide-types -mmcu=atmega328p -DF_CPU=20000000L  '-DLED_START_FLASHES=3' '-DBAUD_RATE=115200' -Wl,--section-start=.text=0x7e00 -Wl,--section-start=.version=0x7ffe -Wl,--relax -Wl,--gc-sections -nostartfiles -nostdlib -o optiboot_atmega328_20.elf optiboot.o
avr-size optiboot_atmega328_20.elf
   text    data     bss     dec     hex filename
    502       0       0     502     1f6 optiboot_atmega328_20.elf
avr-objcopy -j .text -j .data -j .version --set-section-flags .version=alloc,load -O ihex optiboot_atmega328_20.elf optiboot_atmega328_20.hex
avr-objdump -h -S optiboot_atmega328_20.elf > optiboot_atmega328_20.lst

rm optiboot_atmega328_20.elf optiboot.o

Two files appeared in optiboot directory: optiboot_atmega328_20.hex, optiboot_atmega328_20.lst

Size of my bootloader (hex) is 1471 bytes.

Now we need add our 20MHz board to Arduino IDE.

Open c:\Program Files (x86)\Arduino\hardware\arduino\avr\boards.txt and append it with

###############################################
uno_20.name=Arduino Uno clone 20MHz

uno_20.upload.tool=avrdude
uno_20.upload.protocol=arduino
uno_20.upload.maximum_size=32256
uno_20.upload.maximum_data_size=2048
uno_20.upload.speed=115200

uno_20.bootloader.tool=avrdude
uno_20.bootloader.low_fuses=0xFF
uno_20.bootloader.high_fuses=0xDE
uno_20.bootloader.extended_fuses=0xfd
uno_20.bootloader.unlock_bits=0x3F
uno_20.bootloader.lock_bits=0x0F
uno_20.bootloader.file=optiboot/optiboot_atmega328_20.hex

uno_20.build.mcu=atmega328p
uno_20.build.f_cpu=20000000L
uno_20.build.board=AVR_UNO
uno_20.build.core=arduino
uno_20.build.variant=standard

Open (restart) Arduino IDE and choose your board from menu.

Try to load bootloader. Chose your ISP programmer (USBasp) first.

Bootloader must successfully loaded and verified.

Then try load your empty sketch by USB-UART (COM port).

4. Connecting ESP-01 and Arduino.

Go to ESP-LINK by web browser. If you use it as AP go to http://192.168.4.1/

On home page set your ESP board in Pin assignment. Press Change.

Actually we must connect just few pins:

ESP           Arduino
RX          -    TX
TX          -    RX
GPIO0    -  Reset
GND.     -    GND

But when Arduino works with 5V levels I recommend use voltage divider on ESP RX pin (see blue wire).

You may use 2-50K Ohm resistors on both sides. It must be around equal on both sides for around a half voltage dividing. 5V/2 = 2.5V. It's around 3V :-) and it's High (not Low) level for ESP.

Also I connect pullup resistor (10K Ohm) to Arduino Reset pin and +3.3V. I think it prevent random 5V to ESP GPOI0.

Now you can add custom programmer to Arduino IDE

Open c:\Program Files (x86)\Arduino\hardware\arduino\avr\programmers.txt and append with:

####################

#ESP-LINK

esp-link.name=ESP-Link

esp-link.communication=serial

esp-link.protocol=arduino

esp-link.program.protocol=arduino

esp-link.program.tool=avrdude

esp-link.program.extra_params=-Pnet:ESP-01-LINK:2323 -DV 

Edit your system hosts file for resolve name ESP-01-LINK with IP of your ESP-LINK
C:\Windows\System32\drivers\etc\hosts

192.168.137.96 ESP-01-LINK

Go to ESP-LINK by web browser. Go to uC Console page and set Baudrate to 115200 or which you set in bootloader config.

Restart Arduino IDE, choose your custom programmer ESP-Link from Tools -> Programmer menu

Try to load sketch using programmer.

Using UART over Wi-Fi is not good idea and. Try few times.

You may enable Arduino IDE upload debug in Arduino settings.

And it's very bad idea if you serial console in you sketches.
Out from console delaying and some bits arrive on port after Arduino actually resets.
It create problems with flashing.

But ESP-LINK have measure to treat it.

5. ESP-LINK Curl programmer

We can upload our sketch by HTTP and just make POST request for reset our Arduino. ESP-LINK package have good shell script for Linux.

Looking into this script I write very simpliest version for Windows.

First of all install Curl for windows
Extract and put dirs in WinAVR directory, just prevent adding to PATH new dir and reboot.

Test it from cmd shell.
curl -v

Create file avrflash.cmd, for example in c:\Program Files (x86)\Arduino\hardware\arduino\avr\

@echo off
curl -m 10 -s -XPOST http://esp-01-link/pgm/sync
sleep 1
curl -m 10 -s http://esp-01-link/pgm/sync
echo.
curl -m 10 -s -g -d "@%*" "http://esp-01-link/pgm/upload"

Then append c:\Program Files (x86)\Arduino\hardware\arduino\avr\platforms.txt with:

######################
# ESP-LINK
tools.esplink.path={runtime.tools.avrdude.path}
tools.esplink.cmd.path={path}/bin/avrdude
tools.esplink.config.path={path}/etc/avrdude.conf

tools.esplink.upload.params.verbose=-v
tools.esplink.upload.params.quiet=-q -q
# tools.esplink.upload.verify is needed for backwards compatibility with IDE 1.6.8 or older, IDE 1.6.9 or newer overrides this value
tools.esplink.upload.verify=
tools.esplink.upload.params.noverify=-V
tools.esplink.upload.pattern="c:\Program Files (x86)\Arduino\hardware\arduino\avr\avrflash.cmd" {build.path}/{build.project_name}.hex

tools.esplink.program.params.verbose=-v
tools.esplink.program.params.quiet=-q -q
# tools.esplink.program.verify is needed for backwards compatibility with IDE 1.6.8 or older, IDE 1.6.9 or newer overrides this value
tools.esplink.program.verify=
tools.esplink.program.params.noverify=-V
tools.esplink.program.pattern="{cmd.path}" "-C{config.path}" {program.verbose} {program.verify} -p{build.mcu} -c{protocol} {program.extra_params} "-Uflash:w:{build.path}/{build.project_name}.hex:i"

tools.esplink.erase.params.verbose=-v
tools.esplink.erase.params.quiet=-q -q
tools.esplink.erase.pattern="{cmd.path}" "-C{config.path}" {erase.verbose} -p{build.mcu} -c{protocol} {program.extra_params} -e -Ulock:w:{bootloader.unlock_bits}:m -Uefuse:w:{bootloader.extended_fuses}:m -Uhfuse:w:{bootloader.high_fuses}:m -Ulfuse:w:{bootloader.low_fuses}:m

tools.esplink.bootloader.params.verbose=-v
tools.esplink.bootloader.params.quiet=-q -q
tools.esplink.bootloader.pattern="{cmd.path}" "-C{config.path}" {bootloader.verbose} -p{build.mcu} -c{protocol} {program.extra_params} "-Uflash:w:{runtime.platform.path}/bootloaders/{bootloader.file}:i" -Ulock:w:{bootloader.lock_bits}:m

Actually we need only bold string, but other just for Arduino compatibility.

Edit our board in boards.txt, change upload tool to esp-link:
uno_20.upload.tool=esplink

Restart Arduino IDE and try to upload you sketch!

Also you can debug your board by open telnet on esp-01-link on port 23 or 2323.
telnet esp-01-link

or use Putty or another telnet terminal.

 

Don't forget to set bitrate to 115200 in esp-link console settings. It help to avoid errors while flashing via curl.