Initial commit

3 files changed, 501 insertions, 0 deletions

diff --git a/README.html b/README.html
new file mode 100644
index 0000000..6029120
--- /dev/null
+++ b/README.html
@@ -0,0 +1,102 @@
+<h1>dcf77_gps for nodemcu</h1>
+<p>
+ This is a lua program intended to be run on an ESP8266 loaded with
+ <a href="https://www.nodemcu.com/">nodemcu</a>. It will parse pulses from a
+ <a href="https://www.pollin.de/p/dcf-77-empfangsmodul-dcf1-810054">pollin</a>
+ <a href="https://en.wikipedia.org/wiki/DCF77">DCF77</a> receiver module. It's
+ output on the serial port (usually available over USB) will be
+ <a href="https://en.wikipedia.org/wiki/NMEA_0183">NMEA GPZDA</a> sentences
+ which can be understood by <a href="https://ntp.org">ntpd</a>.
+</p>
+<h2>Why this program?</h2>
+<p>
+ First of all i wanted to learn more about DCF77 and gain experience
+ programming in lua. I also wanted to connect DCF77 to my (Raspberry Pi 2B
+ based) ntp server without compromising the accuracy of it's already available
+ PPS signal. When the receiver module looses it's signal it can easily generate
+ hunderds or thousands of interrupts a second which will definately harm the
+ PPS accuracy. By moving the processing to an ESP8266 the ESP will take the hit
+ and just output valid data when it has a decent signal.
+</p>
+<h2>What does it do?</h2>
+<p>
+ It receives data from a DCF77 module and outputs the same data as a GPS would
+ do. To do that, it has some (IMHO) neat tricks. Usually it will run in an
+ interrupt driven way, with a clean signal this will trigger twice a second
+ (one for the rising edge, one for the falling edge). When it detects way more
+ interrupts than expected, it switches to polling mode. It will use polling to
+ detect a clean signal, and only when it's clean enough it will change back to
+ interrupt mode again.
+</p>
+<p>
+ To improve accuracy and reduce the time from the rising edge of the radio 
+ signal to the delivery of the GPS sentence over the serial port it partially
+ calculates the required checksum for the sentences every minute. Doing so only
+ leaves the second characters to be added to the checksum.
+</p>
+<p>
+ When a clean signal is received it will blink a led to let you know it's
+ working. Because LED's can be very bright when it gets dark you can set a
+ minimum ambient light which will be read from the adc. When the ambient light
+ is lower it will stop blinking.
+</p>
+<h2>How can i use this?</h2>
+<p>
+ Of course you'll need an ESP8266. The ESP32 might actually be compatible, but
+ i don't know and haven't tested. And of course you'll need a DCF77 receiver
+ module. The pollin one i have is pretty cheap (around
+ <a href="https://www.vandijkenelektronica.nl/product/dcf77-moduul-met-ferrietantenne-voor-de-dcf-77-uitlezing/">EUR 10</a>,
+ ignore the image on the site, it's the conrad module, but you'll get the
+ pollin one when you order), the popular conrad module might also work but
+ again: I haven't tested it.
+</p>
+<p>
+ The nodemcu firmware can be build using
+ <a href="https://nodemcu-build.com/">their build service</a>. This program
+ requires the <b>adc</b>, <b>bit</b>, <b>gpio</b> and <b>timer</b> modules. If
+ you want to use 
+ <a href="https://github.com/kmpm/nodemcu-uploader">nodemcu-uploader</a> to
+ upload the program, be aware that you will also need the <b>file</b>,
+ <b>node</b> and <b>uart</b> modules.
+</p>
+<p>
+ To use the program upload it to your ESP8266 and if you want it to start
+ automatically when the ESP is started also upload the init.lua file. Restart
+ the ESP3288 and then wait :-)
+</p>
+<p>
+ If you look at the code you'll see quite some print statements commented out,
+ those are there to help debugging. Unfortunately memory on the ESP8266 is too
+ constrained to be able to uncomment them all at the same time, so use with
+ caution.
+</p>
+<h2>What can do with this?</h2>
+<p>
+ With this you can make ntpd think it's receiving from a GPS while it's
+ actually using your DCF77 receiver. To do that, first you'll have to instruct
+ udev to create a gps symlink. I use the following rules in
+ /etc/udev/rules.d/99-gps.rules:
+</p>
+<pre>
+KERNEL=="ttyUSB0", SYMLINK+="gps0"
+</pre>
+<p>
+ After that configure ntpd to use it with a server and fudge line in
+ /etc/ntp.conf:
+</p>
+<pre>
+server 127.127.20.0 mode 0x58 minpoll 3 maxpoll 3 prefer
+fudge 127.127.20.0 stratum 0 flag1 0 flag2 0 flag3 0 flag4 0 time1 0.0 time2 0.0292 refid DCF0
+</pre>
+<p>
+ Especially time2 should be tuned to your own setup. The command <b>ntpq
+ -pn</b> should now show something like this:
+</p>
+<pre>
+     remote           refid      st t when poll reach   delay   offset  jitter
+==============================================================================
+*127.127.20.0    .DCF0.           0 l    4    8  377    0.000   -0.214   0.567
+</pre>
+<p>
+ Have fun!
+</p>
diff --git a/dcf77_gps.lua b/dcf77_gps.lua
new file mode 100644
index 0000000..e9f24f1
--- /dev/null
+++ b/dcf77_gps.lua
@@ -0,0 +1,397 @@
+dcf77_pin=6
+led_pin=4
+
+min_ambient_light=25
+
+last_valid_rise=0
+
+valid_rise_min=990000
+valid_rise_max=1010000
+valid_timeout=5000000
+
+valid_newmin_min=1990000
+valid_newmin_max=2100000
+
+valid_0_min=100000
+valid_0_max=140000
+valid_1_min=200000
+valid_1_max=250000
+
+dcfdata=""
+
+valid_times=0
+total_times=0
+
+bcd={}
+bcd[1]=1
+bcd[2]=2
+bcd[3]=4
+bcd[4]=8
+bcd[5]=10
+bcd[6]=20
+bcd[7]=40
+bcd[8]=80
+
+dow={}
+dow[0]="Invalid"
+dow[1]="Mon"
+dow[2]="Tue"
+dow[3]="Wed"
+dow[4]="Thu"
+dow[5]="Fri"
+dow[6]="Sat"
+dow[7]="Sun"
+
+mon_days={}
+mon_days[1]=31
+mon_days[2]=28
+mon_days[3]=31
+mon_days[4]=30
+mon_days[5]=31
+mon_days[6]=30
+mon_days[7]=31
+mon_days[8]=31
+mon_days[9]=30
+mon_days[10]=31
+mon_days[11]=30
+mon_days[12]=31
+
+function bcd2dec(data)
+ local value=0
+ for count=1,string.len(data),1
+ do
+  if string.sub(data,count,count)=="1" and bcd[count]~=nil then
+   value=value+bcd[count]
+  end
+ end
+ return value
+end
+
+function check_parity(data)
+ local onecount=0
+ for count=1,string.len(data),1
+ do
+  if string.sub(data,count,count)=="1" then
+   onecount=onecount+1
+  end
+ end
+ if onecount%2==0 then
+  return true
+ end
+ return false
+end
+
+gps_sec=0
+gps_valid=false
+gps_part1=""
+gps_part2=""
+gps_part_checksum=0
+
+hex={}
+hex[0]="0"
+hex[1]="1"
+hex[2]="2"
+hex[3]="3"
+hex[4]="4"
+hex[5]="5"
+hex[6]="6"
+hex[7]="7"
+hex[8]="8"
+hex[9]="9"
+hex[10]="A"
+hex[11]="B"
+hex[12]="C"
+hex[13]="D"
+hex[14]="E"
+hex[15]="F"
+
+function send_gpsstring()
+ local gps_data
+ local gps_checksum=gps_part_checksum
+ local part1len=string.len(gps_part1)
+
+ if gps_sec<10 then
+  gps_data=gps_part1.."0"..gps_sec..gps_part2
+ else
+  gps_data=gps_part1..gps_sec..gps_part2
+ end
+
+ gps_checksum=bit.bxor(gps_checksum,string.byte(string.sub(gps_data,part1len+1,part1len+1)))
+ gps_checksum=bit.bxor(gps_checksum,string.byte(string.sub(gps_data,part1len+2,part1len+2)))
+
+ local digit1=bit.rshift(gps_checksum,4)
+ local digit2=bit.band(gps_checksum,15)
+
+ print("$"..gps_data.."*"..hex[digit1]..hex[digit2].."\r")
+end
+
+function parse_data(data)
+ total_times=total_times+1
+ --print("-------------------------------------------------------------")
+ --print(data)
+ --print("Resrv: ",string.sub(data,1,15))
+ --print("Info: ",string.sub(data,16,21))
+ --print("Min: ",string.sub(data,22,29),bcd2dec(string.sub(data,22,28)),check_parity(string.sub(data,22,29)))
+ --print("Hour: ",string.sub(data,30,36).." ",bcd2dec(string.sub(data,30,35)),check_parity(string.sub(data,30,36)))
+ --print("Day: ",string.sub(data,37,42).."  ",bcd2dec(string.sub(data,37,42)))
+ --print("DOW: ",string.sub(data,43,45).."     ",bcd2dec(string.sub(data,43,45)))
+ --print("Month: ",string.sub(data,46,50).."    ",bcd2dec(string.sub(data,46,50)))
+ --print("Year: ",string.sub(data,51,59),bcd2dec(string.sub(data,51,59)),check_parity(string.sub(data,37,59)))
+ --print("Rest: ",string.sub(data,60))
+ --if string.len(data)==59 then
+ -- print("Time/date : "..bcd2dec(string.sub(data,30,35))..":"..bcd2dec(string.sub(data,22,28)).." "..dow[bcd2dec(string.sub(data,43,45))].." 20"..bcd2dec(string.sub(data,51,59)).."-"..bcd2dec(string.sub(data,46,50)).."-"..bcd2dec(string.sub(data,37,42)))
+ --end
+
+ gps_valid=false
+
+ local min
+ local hour
+ local day
+ local mon
+ local year
+
+ if data==nil then
+  --print("No data received")
+  return
+ end
+
+ if string.len(data)~=59 then
+  --print("Data length invalid")
+  return
+ end
+ --print("Data length valid")
+
+ if string.sub(data,18,18)==string.sub(data,19,19) then
+  --print("Timezone data is invalid")
+  return
+ end
+ --print("Timezone data is valid")
+
+ if string.sub(data,21,21)=="0" then
+  --print("Start time bit is invalid")
+  return
+ end
+ --print("Start time bit is valid")
+
+ if not check_parity(string.sub(data,22,29)) then
+  --print("Minute parity invalid")
+  return
+ end
+ --print("Minute parity valid")
+ min=bcd2dec(string.sub(data,22,28))
+ if min>59 then
+  --print("Minute value invalid")
+  return
+ end
+ --print("Minute value valid")
+
+ if not check_parity(string.sub(data,30,36)) then
+  --print("Hour parity invalid")
+  return
+ end
+ --print("Hour parity valid")
+ hour=bcd2dec(string.sub(data,30,35))
+ if hour>23 then
+  --print("Hour valid invalid")
+  return
+ end
+ --print("Hour valid valid")
+
+ if not check_parity(string.sub(data,37,59)) then
+  --print("Date parity invalid")
+  return
+ end
+ --print("Date parity valid")
+
+ day=bcd2dec(string.sub(data,37,42))
+ if day<1 or day>31 then
+  --print("Day value invalid")
+  return
+ end
+ --print("Day valud valid")
+
+ mon=bcd2dec(string.sub(data,46,50))
+ if mon<1 or mon>12 then
+  --print("Month value invalid")
+  return
+ end
+ --print("Month value valid")
+
+ year=bcd2dec(string.sub(data,51,59))
+ if year<22 or year>30 then
+  --print("Year value invalid")
+  return
+ end
+ --print("Year value valid")
+
+ valid_times=valid_times+1
+ --print("Total "..valid_times.." valid times and "..total_times-valid_times.." invalid times")
+
+ if string.sub(data,18,18)==1 then
+  --print("Subtracting 1 hour for CET timezone")
+  hour=hour-1
+ else
+  --print("Subtracting 2 hours for CEST timezone")
+  hour=hour-2
+ end
+ if hour<0 then
+  hour=hour+24
+  day=day-1
+  if day<1 then
+   mon=mon-1
+   if mon<1 then
+    mon=12
+    year=year-1
+   end
+   if mon==2 and year%4==0 then
+    day=day+1
+   end
+   day=mon_days[mon]
+  end
+ end
+ gps_part1=string.format("GPZDA,%02d%02d",hour,min)
+ gps_part2=string.format(".000,%02d,%02d,20%02d,,",day,mon,year)
+
+ local gps_data=gps_part1..gps_part2
+ gps_part_checksum=0
+
+ for count=1,string.len(gps_data)
+ do
+  gps_part_checksum=bit.bxor(gps_part_checksum,string.byte(string.sub(gps_data,count,count)))
+ end
+
+ gps_sec=0
+ gps_valid=true
+ send_gpsstring()
+end
+
+function irq_handler(level,when,count)
+ if count>10 then
+  --print("Waiting for stable signal, changing to polling")
+  start_poll()
+  return
+ end
+ if when<last_valid_rise then
+  delta=(2147483647-last_valid_rise)+when
+ else
+  delta=when-last_valid_rise
+ end
+ if level==1 then
+  --print("Rise delta: "..delta)
+  if last_valid_rise==0 then
+   last_valid_rise=when
+   return
+  end
+  if delta>valid_newmin_min and delta<valid_newmin_max then
+   parse_data(dcfdata)
+   dcfdata=""
+   last_valid_rise=when
+   return
+  end
+  if delta>valid_rise_min and delta<valid_rise_max then
+   --print("valid")
+   last_valid_rise=when
+   if gps_valid==true then
+    gps_sec=gps_sec+1
+    if gps_sec<60 then
+     send_gpsstring()
+     if adc.read(0)>min_ambient_light then
+      gpio.write(led_pin,gpio.LOW)
+     end
+    else
+     gps_valid=false
+     gpio.write(led_pin,gpio.HIGH)
+    end
+   end
+   return
+  end
+  if delta>valid_timeout then
+   --print("Valid timeout")
+   last_valid_rise=0
+   return
+  end
+ end
+ if level==0 then
+  --print("Pulse length: "..delta)
+  if delta>valid_0_min and delta<valid_0_max then
+   --print("0")
+   dcfdata=dcfdata.."0"
+   gpio.write(led_pin,gpio.HIGH)
+  end
+  if delta>valid_1_min and delta<valid_1_max then
+   --print("1")
+   dcfdata=dcfdata.."1"
+   gpio.write(led_pin,gpio.HIGH)
+  end
+  if string.len(dcfdata)>59 then
+   dcfdata=string.sub(dcfdata,2)
+  end
+ end
+end
+
+function irq_dummy(level,when,count)
+ --print("Dummy IRQ handler")
+ return
+end
+
+poll_prev_value=0
+poll_value_count={}
+poll_value_count[0]=0
+poll_value_count[1]=0
+poll_clean=0
+
+function poll_handler()
+ local value
+ value=gpio.read(dcf77_pin)
+ if value==nil then
+  poll_timer:start()
+ end
+ if value==1 and poll_prev_value==0 then
+  --print(poll_value_count[0],poll_value_count[1])
+  if poll_value_count[0]>70 and poll_value_count[0]<100 and poll_value_count[1]>6 and poll_value_count[1]<30 then
+   poll_clean=poll_clean+1
+   --print(poll_clean.." clean pulses")
+   if poll_clean>10 then
+    poll_clean=0
+    --print("Clean signal, swithcing to IRQ handling")
+    start_irq()
+    return
+   end
+  else
+   poll_clean=0
+  end
+  poll_value_count[0]=0
+  poll_value_count[1]=0
+ end
+ poll_prev_value=value
+ poll_value_count[value]=poll_value_count[value]+1
+ poll_timer:start()
+end
+
+poll_timer=tmr.create()
+poll_timer:register(10, tmr.ALARM_SEMI,poll_handler)
+
+function start_poll()
+ gpio.write(led_pin,gpio.HIGH)
+ gpio.trig(dcf77_pin,"none",irq_dummy)
+ gpio.mode(dcf77_pin,gpio.INPUT)
+ poll_timer:start()
+end
+
+function start_irq()
+ gpio.mode(led_pin,gpio.OUTPUT)
+ gpio.write(led_pin,gpio.HIGH)
+ gps_valid=false
+ gpio.trig(dcf77_pin,"both",irq_handler)
+ gpio.mode(dcf77_pin,gpio.INT)
+end
+
+function start_dcf77()
+ start_irq()
+end
+
+function stop_dcf77()
+ gpio.mode(dcf77_pin,gpio.INPUT)
+ poll_timer:stop()
+ gpio.write(led_pin,gpio.HIGH)
+end
diff --git a/init.lua b/init.lua
new file mode 100644
index 0000000..6b21b2b
--- /dev/null
+++ b/init.lua
@@ -0,0 +1,2 @@
+dofile("dcf77_gps.lua")
+start_dcf77()