20190705_jeffrey - silenceuncrio/diary GitHub Wiki

0905

繼續 M360 - feature/lighttpd

回顧昨天遇到的問題

openwrt/packages/net/lighttpd 版本是 1.4.53

make[4]: Entering directory `/home/user/build_dir/target-mipsel_24kec+dsp_uClibc-0.9.33.2/lighttpd-1.4.53'
 cd . && /bin/bash /home/user/build_dir/target-mipsel_24kec+dsp_uClibc-0.9.33.2/lighttpd-1.4.53/missing --run automake-1.11 --foreign
configure.ac:24: require Automake 1.13, but have 1.11.6
make[4]: *** [Makefile.in] Error 1

來自 mistrastar 的 toolchain 裡的 automake 版本太舊...

改從 M330 搬現成的 package

刪掉目前的 package/net/lighttpd

從 M330 複製 lighttpd package

➜  lighttpd git:(develop) ✗ cd ~/M360P
➜  M360P git:(develop) ✗ rm -rf package/net/lighttpd
➜  M360P git:(develop) ✗ cp -r ~/m330/feeds/packages/net/lighttpd package/net
➜  M360P git:(develop) ✗ ls -al package/net/
total 48
drwxrwxr-x 12 jeffrey jeffrey 4096  七   5 09:28 .
drwxrwxr-x 19 jeffrey jeffrey 4096  六   5  2018 ..
drwxrwxr-x  3 jeffrey jeffrey 4096  六   5  2018 bridge-utils
drwxrwxr-x  4 jeffrey jeffrey 4096 十一 16  2018 isc-dhcp
drwxrwxr-x  4 jeffrey jeffrey 4096  六   5  2018 keepalived
drwxrwxr-x  3 jeffrey jeffrey 4096  七   5 09:28 lighttpd
drwxrwxr-x  4 jeffrey jeffrey 4096  六   5  2018 mosquitto
drwxrwxr-x  4 jeffrey jeffrey 4096  六   5  2018 net-snmp
drwxrwxr-x  6 jeffrey jeffrey 4096  六  25 14:00 nginx
drwxrwxr-x  4 jeffrey jeffrey 4096  六   5  2018 quagga
drwxrwxr-x  4 jeffrey jeffrey 4096  六   5  2018 strongswan
drwxrwxr-x  3 jeffrey jeffrey 4096  六   5  2018 xl2tpd
➜  M360P git:(develop) ✗

退出 M360P build 的 docker container 後再進入

➜  ~ ./360.sh
user@10972b0a698a:/$ cd /home/user/
user@10972b0a698a:~$ source proenv.sh
...
#
# configuration written to .config
#
user@10972b0a698a:~$ make menuconfig

發現 lighttpd 在 Network > Web Servers/Proxies > lighttpd

參考 20190619_jeffrey M330 的選項

SSL support
lighttpd-mod-cgi - CGI module
lighttpd-mod-rewrite - URL rewriting module

--- lighttpd........................... A flexible and lightweight web server        
[*]   SSL support (NEW)                                                              
< >   lighttpd-mod-access........................... Access restrictions module (NEW)
< >   lighttpd-mod-accesslog............................. Access logging module (NEW)
< >   lighttpd-mod-alias................................ Directory alias module (NEW)
< >   lighttpd-mod-auth.................................. Authentication module (NEW)
<*>   lighttpd-mod-cgi.............................................. CGI module      
< >   lighttpd-mod-cml.............................. Cache Meta Language module (NEW)
< >   lighttpd-mod-compress............................. Compress output module (NEW)
< >   lighttpd-mod-evasive...................................... Evasive module (NEW)
< >   lighttpd-mod-evhost..................... Exnhanced Virtual-Hosting module (NEW)
< >   lighttpd-mod-expire........................................ Expire module (NEW)
< >   lighttpd-mod-extforward......................... Extract client IP module (NEW)
< >   lighttpd-mod-fastcgi...................................... FastCGI module (NEW)
< >   lighttpd-mod-flv-streaming.......................... flv streaming module (NEW)
< >   lighttpd-mod-magnet........................................ magnet module (NEW)
< >   lighttpd-mod-proxy.......................................... Proxy module (NEW)
< >   lighttpd-mod-redirect............................. URL redirection module (NEW)
<*>   lighttpd-mod-rewrite................................ URL rewriting module      
< >   lighttpd-mod-rrdtool...................................... rrdtool module (NEW)
< >   lighttpd-mod-scgi............................................ SCGI module (NEW)
< >   lighttpd-mod-secdownload................. Secure and fast download module (NEW)
< >   lighttpd-mod-setenv.................. Environment variable setting module (NEW)
< >   lighttpd-mod-simple-vhost.................. Simple virtual hosting module (NEW)
< >   lighttpd-mod-ssi.............................................. SSI module (NEW)
< >   lighttpd-mod-status......................... Server status display module (NEW)
< >   lighttpd-mod-trigger-b4-dl................ Trigger before download module (NEW)
< >   lighttpd-mod-userdir............................... User directory module (NEW)
< >   lighttpd-mod-usertrack.............................. User tracking module (NEW)
< >   lighttpd-mod-webdav........................................ WebDAV module (NEW)

存檔並退出 OpenWrt Configuration 我們在最外層可以從 .config 看到 lighttpd 相關的 compiler flag

#
# Web Servers/Proxies
#
CONFIG_PACKAGE_lighttpd=y
CONFIG_LIGHTTPD_SSL=y
# CONFIG_PACKAGE_lighttpd-mod-access is not set
# CONFIG_PACKAGE_lighttpd-mod-accesslog is not set
# CONFIG_PACKAGE_lighttpd-mod-alias is not set
# CONFIG_PACKAGE_lighttpd-mod-auth is not set
CONFIG_PACKAGE_lighttpd-mod-cgi=y
# CONFIG_PACKAGE_lighttpd-mod-cml is not set
# CONFIG_PACKAGE_lighttpd-mod-compress is not set
# CONFIG_PACKAGE_lighttpd-mod-evasive is not set
# CONFIG_PACKAGE_lighttpd-mod-evhost is not set
# CONFIG_PACKAGE_lighttpd-mod-expire is not set
# CONFIG_PACKAGE_lighttpd-mod-extforward is not set
# CONFIG_PACKAGE_lighttpd-mod-fastcgi is not set
# CONFIG_PACKAGE_lighttpd-mod-flv-streaming is not set
# CONFIG_PACKAGE_lighttpd-mod-magnet is not set
# CONFIG_PACKAGE_lighttpd-mod-proxy is not set
# CONFIG_PACKAGE_lighttpd-mod-redirect is not set
CONFIG_PACKAGE_lighttpd-mod-rewrite=y
# CONFIG_PACKAGE_lighttpd-mod-rrdtool is not set
# CONFIG_PACKAGE_lighttpd-mod-scgi is not set
# CONFIG_PACKAGE_lighttpd-mod-secdownload is not set
# CONFIG_PACKAGE_lighttpd-mod-setenv is not set
# CONFIG_PACKAGE_lighttpd-mod-simple-vhost is not set
# CONFIG_PACKAGE_lighttpd-mod-ssi is not set
# CONFIG_PACKAGE_lighttpd-mod-status is not set
# CONFIG_PACKAGE_lighttpd-mod-trigger-b4-dl is not set
# CONFIG_PACKAGE_lighttpd-mod-userdir is not set
# CONFIG_PACKAGE_lighttpd-mod-usertrack is not set
# CONFIG_PACKAGE_lighttpd-mod-webdav is not set
# CONFIG_PACKAGE_nginx-all-module is not set
CONFIG_PACKAGE_nginx-ssl=y

參考該檔案來修改 m360p_defconfig

diff --git a/m360p_defconfig b/m360p_defconfig
index 64e6cf9..c58e369 100644
--- a/m360p_defconfig
+++ b/m360p_defconfig
@@ -3241,6 +3241,10 @@ CONFIG_PACKAGE_xl2tpd=y
 #
 # Web Servers/Proxies
 #
+CONFIG_PACKAGE_lighttpd=y
+CONFIG_LIGHTTPD_SSL=y
+CONFIG_PACKAGE_lighttpd-mod-cgi=y
+CONFIG_PACKAGE_lighttpd-mod-rewrite=y
 # CONFIG_PACKAGE_sockd is not set
 # CONFIG_PACKAGE_socksify is not set
 CONFIG_PACKAGE_uhttpd=y

退出 M360P build 的 docker container 後再進入

➜  ~ ./360.sh
user@10972b0a698a:/$ cd /home/user/
user@10972b0a698a:~$ source proenv.sh
...
#
# configuration written to .config
#
user@10972b0a698a:~$ make V=s

等吧

0955

應該是 build 過了

進 proscend 目錄

user@10972b0a698a:~/proscend$ make distclean
...
user@10972b0a698a:~/proscend$ make menuconfig
...

Products/Vendor = M360P/2_PI

user@10972b0a698a:~/proscend$ make clean
...
user@10972b0a698a:~/proscend$ make
...

先去開 BU II 周會

1120

image 已經產生了

透過 uboot upgrade image 吧

root@M360-P:/home/admin# lighttpd -h
lighttpd/1.4.30 (ssl) (Jul  5 2019 01:41:50) - a light and fast webserver
usage:
 -f <name>  filename of the config-file
 -m <name>  module directory (default: /usr/lib/lighttpd)
 -p         print the parsed config-file in internal form, and exit
 -t         test the config-file, and exit
 -D         don't go to background (default: go to background)
 -v         show version
 -V         show compile-time features
 -h         show this help

非常好

終於在 M360P 踏出了第一步

1305

幫忙回答 GRE 相關的客戶提問

支援GRE tunnel需支援Keep alive、Key加密、上下行Bundle rate saping

review 目前我們的 GRE 功能之後以及 google 客戶的提問後回答如下

Dear Ariel,

我們的 GRE
沒有支援 keep alive
沒有 Key加密

至於上下行Bundle rate saping
我不了解 saping 是什麼
不過我們 GRE 也沒有提供跟 rate 設定有關的參數

BR Jeffrey

1320

先幫 M360 - feature/lighttpd 的第一步作 commit

➜  M360P git:(develop) ✗ git stash
Saved working directory and index state WIP on develop: fbf3127 Merge tag 'v1.03-patch' into develop
HEAD is now at fbf3127 Merge tag 'v1.03-patch' into develop
➜  M360P git:(develop) ✗ git flow feature start lighttpd
Fatal: Not a gitflow-enabled repo yet. Please run 'git flow init' first.
➜  M360P git:(develop) ✗ git flow init
...
➜  M360P git:(develop) ✗ git flow feature start lighttpd
Switched to a new branch 'feature/lighttpd'

Summary of actions:
- A new branch 'feature/lighttpd' was created, based on 'develop'
- You are now on branch 'feature/lighttpd'

Now, start committing on your feature. When done, use:

     git flow feature finish lighttpd

➜  M360P git:(feature/lighttpd) ✗ git stash apply
On branch feature/lighttpd
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

        modified:   m360p_defconfig

Untracked files:
  (use "git add <file>..." to include in what will be committed)

        package/net/lighttpd/
        proscend/mconfig/lxdialog/lxdialog
        proscend/prosrc/icos/rawpacket/rawping

no changes added to commit (use "git add" and/or "git commit -a")
➜  M360P git:(feature/lighttpd) ✗ git status
On branch feature/lighttpd
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

        modified:   m360p_defconfig

Untracked files:
  (use "git add <file>..." to include in what will be committed)

        package/net/lighttpd/
        proscend/mconfig/lxdialog/lxdialog
        proscend/prosrc/icos/rawpacket/rawping

no changes added to commit (use "git add" and/or "git commit -a")

commit 並 publish

➜  M360P git:(feature/lighttpd) ✗ git flow feature publish
Counting objects: 10, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (10/10), done.
Writing objects: 100% (10/10), 5.30 KiB | 0 bytes/s, done.
Total 10 (delta 4), reused 0 (delta 0)
remote:
remote: To create a merge request for feature/lighttpd, visit:
remote:   https://192.168.0.124/RD/M360P/merge_requests/new?merge_request%5Bsource_branch%5D=feature%2Flighttpd
remote:
To [email protected]:RD/M360P.git
 * [new branch]      feature/lighttpd -> feature/lighttpd
Branch feature/lighttpd set up to track remote branch feature/lighttpd from origin.
Already on 'feature/lighttpd'
Your branch is up-to-date with 'origin/feature/lighttpd'.

Summary of actions:
- The remote branch 'feature/lighttpd' was created or updated
- The local branch 'feature/lighttpd' was configured to track the remote branch
- You are now on branch 'feature/lighttpd'

M360P - feature/lighttpd - use lighttpd to replace the current web server iweb(mongoose)

commit e3342ad7f14c1a75452753923334b7d353222d76
Refs: [feature/lighttpd], {origin/feature/lighttpd}
Author: jeffrey <[email protected]>
Date:   Fri Jul 5 13:27:03 2019 +0800

    feature/lighttpd - use lighttpd to replace the current web server iweb(mongoose)
    - lighttpd/1.4.30 (ssl) - a light and fast webserver
    - lighttpd - A flexible and lightweight web server
      - SSL support
      - lighttpd-mod-cgi - CGI module
      - lighttpd-mod-rewrite - URL rewriting module

 m360p_defconfig                          |   4 +
 package/net/lighttpd/Makefile            | 327 +++++++++++++++++++++++++++++++
 package/net/lighttpd/files/lighttpd.conf | 231 ++++++++++++++++++++++
 package/net/lighttpd/files/lighttpd.init |  16 ++
 4 files changed, 578 insertions(+)

再來就 follow M330 的步調前進吧

1350

OpenWrt 的系統在 /etc/init.d 之下有個 lighttpd 在系統一開機的時候會去啟動

/etc/init.d/lighttpd

#!/bin/sh /etc/rc.common
# Copyright (C) 2006-2011 OpenWrt.org

SERVICE_USE_PID=1

START=50

start() {
        mkdir -m 0755 -p /var/log/lighttpd
        service_start /usr/sbin/lighttpd -f /etc/lighttpd/lighttpd.conf
}

stop() {
        service_stop /usr/sbin/lighttpd
}

這需要利用 M360P source base 下的 proscend/Makefile 來將其刪除

diff --git a/proscend/Makefile b/proscend/Makefile
index ff477cd..fcfe1fb 100644
--- a/proscend/Makefile
+++ b/proscend/Makefile
@@ -164,7 +164,7 @@ prepare: buildenv_check
        @echo "================================================"
        cd rootfs/lib; rm -rf config wifi
        cd rootfs/etc/init.d; rm -f xl2tpd snmpd openvpn pptpd ipsec uhttpd quagga postfix mosquitto qos log keepalived
-       cd rootfs/etc/init.d; rm -f firewall sysntpd network
+       cd rootfs/etc/init.d; rm -f firewall sysntpd network lighttpd
        rm -f rootfs/etc/rc.d/*nginx

 #########################################################################################

build 個 image 試試

/etc/init.d/lighttpd 的確被刪掉了

commit

commit fd05ce1400f4bf821856eede1e09c358745c8edd
Refs: [feature/lighttpd], {origin/feature/lighttpd}
Author: jeffrey <[email protected]>
Date:   Fri Jul 5 14:01:52 2019 +0800

    feature/lighttpd - do not let the system start lighttpd at beginning

 proscend/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

1405

先盤一下這禮拜作了什麼

M360P - hotfix/v1.03-patch - fix bad block problem
M360P - feature/lighttpd - use lighttpd to replace the current web server iweb(mongoose)
M330 - feature/lighttpd
- 'Service > SNMP'
- 'Service > TR069'
- 'Service > Dynamic DNS'
- 'Service > VRRP'
- 'Service > MQTT'
- 'Service > UPnP'
- 'Service > SMTP'
- 'Service > IP Alias'
- 'Management > Identification'
- 'Management > Administration'
- 'Management > SSH'
- 'Management > Firmware'
- 'Management > Configuartion'
- 'Management > Load Factory'
- 'Management > Restart'
- 'Management > Schedule Reboot'
- replace the iweb with lighttpd at icos web module
M330 - develop - sometimes progress bar stop at 99% while firmware upgrade at web ui
M300 - feature/web_ui_intro - web introduction: 03

1505

M330 - develop - sometimes progress bar stop at 99% while firmware upgrade at web ui

jessy 剛剛表示該次的解法無效

在他 upgrade 了 2x 次之後又發生了

修改的想法如下

FirmwareProgress.sh 不再去寫 /tmp/FirmwareUpgrade.progress.json

只讀就好了至少有機會讀到被更新的 /tmp/FirmwareUpgrade.progress.json

那 mutex 也沒必要了

build image

測試吧

20 次看來是不夠的

來個 30 次吧

01 - PASS
02 - PASS
03 - PASS
04 - PASS
05 - PASS
06 - PASS
07 - PASS
08 - PASS
09 - PASS
10 - PASS

commit 上去讓 jessy 也幫忙 test

M330 - develop - sometimes progress bar stop at 99% while firmware upgrade at web ui

commit 637cee1c9dcfed990a195275f970413c5803641e
Refs: [develop]
Author: jeffrey <[email protected]>
Date:   Fri Jul 5 16:11:10 2019 +0800

    sometimes progress bar stop at 99% while firmware upgrade at web ui
    - nomore mutex-like to protect the access to the progress information file whie firmware upgrade
    - only read '/tmp/FirmwareUpgrade.progress.json' from FirmwareProgress.sh

 proscend/prosrc/icos/script/FirmwareProgress.sh | 25 +----------------
 proscend/prosrc/icos/script/FirmwareUpgrade.sh  | 36 -------------------------
 proscend/prosrc/webcgi/firmware.c               |  7 ++---
 3 files changed, 5 insertions(+), 63 deletions(-)

繼續測

11 - PASS
12 - PASS
13 - PASS
14 - PASS
15 - PASS
16 - PASS
17 - PASS
18 - PASS
19 - PASS
20 - PASS
21 - PASS
22 - PASS
23 - PASS
24 - PASS
25 - PASS
26 - PASS
27 - PASS
28 - PASS
29 - PASS
30 - PASS

1700

回到 M360 - feature/lighttpd

1735

目前都需要手動啟動 lighttpd

搭配 tftp 使用的紀錄如下

root@M360-P:/home/admin# cp -r /www/ /tmp/
root@M360-P:/home/admin# cd /tmp/
root@M360-P:/tmp# tftp -g -r lighttpd.conf 192.168.1.113
root@M360-P:/tmp# tftp -g -r web_x509_lighttpd.sh 192.168.1.113
root@M360-P:/tmp# chmod 755 web_x509_lighttpd.sh
root@M360-P:/tmp# ./web_x509_lighttpd.sh
Generating a 2048 bit RSA private key
......................................+++
.........+++
writing new private key to '/tmp/icos/web/lighttpd.pem'
-----
root@M360-P:/tmp# lighttpd -f /tmp/lighttpd.conf
2019-05-24 12:37:26: (log.c.166) server started
root@M360-P:/tmp#

1740

M360 - feature/lighttpd - session management, login, logout and access filter

M360 - feature/lighttpd - feature/lighttpd - 'System > Time and Date'

commit 2cd6ff3991274cac44bf12d89ccdb97748244bf9
Refs: [feature/lighttpd], {origin/feature/lighttpd}
Author: jeffrey <[email protected]>
Date:   Fri Jul 5 17:40:14 2019 +0800

    feature/lighttpd - session management, login, logout and access filter
    - jweb.access.filter
      - for multi-user with different level
      - only one user for M330 - root with level ATTVAL_SYSTEM_LEVEL3
    - jweb.api.login
      - create a new session for the user and send a cookie to the client
      - cgi_init() do 3 more things:
        - icos_users_init()
          - init the user information form MODULE_SYSTEM
        - init_session_store()
          - we need keep session information at ram
        - check_sessions()
          - trigger this at each time any CGI invoked
          - Cleans up sessions that have been idle for too long
      - use UUID library to generate unique session id
    - jweb.api.logout
      - destroy the current session

    feature/lighttpd - 'System > Time and Date'
    - jweb.access.filter(ATTVAL_SYSTEM_LEVEL1, _config);
    - jweb.access.filter(ATTVAL_SYSTEM_LEVEL2, _apply);
    - jweb.access.filter(ATTVAL_SYSTEM_LEVEL1, _help);

 proscend/prosrc/webcgi/Makefile |   9 +-
 proscend/prosrc/webcgi/api.c    |  42 ++++
 proscend/prosrc/webcgi/jweb.c   | 438 +++++++++++++++++++++++++++++++++++++++-
 proscend/prosrc/webcgi/jweb.h   |  67 +++++-
 proscend/prosrc/webcgi/sntp.c   |   8 +-
 5 files changed, 555 insertions(+), 9 deletions(-)

M360P - feature/lighttpd - 'System > Logging'

commit 51b1afcd16a6c009ec263e9fc01cfe954376936a
Refs: [feature/lighttpd], {origin/feature/lighttpd}
Author: jeffrey <[email protected]>
Date:   Fri Jul 5 17:50:07 2019 +0800

    feature/lighttpd - 'System > Logging'

    syslog.cgi
    - jweb.access.filter(ATTVAL_SYSTEM_LEVEL1, _config);
    - jweb.access.filter(ATTVAL_SYSTEM_LEVEL1, _gen_log);
    - jweb.access.filter(ATTVAL_SYSTEM_LEVEL2, _clear_log);
    - jweb.access.filter(ATTVAL_SYSTEM_LEVEL2, _apply);
    - jweb.access.filter(ATTVAL_SYSTEM_LEVEL1, _help);

    use the full path of `ParseSyslog.sh` or cgi with lighttpd will fail

 proscend/prosrc/webcgi/syslog.c | 17 +++++++----------
 1 file changed, 7 insertions(+), 10 deletions(-)

M360P - feature/lighttpd - 'System > Alarm'

commit 3c21038fd715d8b6cab47471e05ff7e2f7e3ffb3
Refs: [feature/lighttpd], {origin/feature/lighttpd}
Author: jeffrey <[email protected]>
Date:   Fri Jul 5 17:58:59 2019 +0800

    feature/lighttpd - 'System > Alarm'

    alarm.cgi
    - jweb.access.filter(ATTVAL_SYSTEM_LEVEL1, act_config);
    - jweb.access.filter(ATTVAL_SYSTEM_LEVEL1, act_users);
    - jweb.access.filter(ATTVAL_SYSTEM_LEVEL1, act_user);
    - jweb.access.filter(ATTVAL_SYSTEM_LEVEL2, act_delete_users);
    - jweb.access.filter(ATTVAL_SYSTEM_LEVEL1, act_groups);
    - jweb.access.filter(ATTVAL_SYSTEM_LEVEL1, act_group);
    - jweb.access.filter(ATTVAL_SYSTEM_LEVEL2, act_delete_group);
    - jweb.access.filter(ATTVAL_SYSTEM_LEVEL1, act_schedules);
    - jweb.access.filter(ATTVAL_SYSTEM_LEVEL1, act_sms);
    - jweb.access.filter(ATTVAL_SYSTEM_LEVEL2, act_apply);
    - jweb.access.filter(ATTVAL_SYSTEM_LEVEL1, _help);

 proscend/prosrc/webcgi/alarm.c | 76 +++++++++++++++++++++++++++++++++++-------
 1 file changed, 64 insertions(+), 12 deletions(-)