superset

开源的BI工具还是选择apache的superset，2021年的是用过davince，结果2023年就不维护了，dataart也是一样的到2023年也没人维护了，dataease国产的人家也要吃饭，社区版也有限制。因而选择用python开发的superset成了唯一的选择了。

1 部署

1.1 docker

vi superset_config.py
import os
SECRET_KEY = 'WeTzvdirVlm8p5zGVGDjEJWO2jnSkT8b1cybtf3DRPgldYsB6Hjd5sCf'
# 启动
docker run -d -p 8088:8088 -v E:/docker/superset/superset_config.py:/app/pythonpath/superset_config.py --name superset apache/superset:GHA-13336228753
# 初始化
## 配置数据库
superset db upgrade
## 初始化
superset init
## 配置FLASK_APP变量
export FLASK_APP=superset
## 配置账号密码，不创建的话，admin的默认密码为admin，这里可以自定义。
flask fab create-admin

1.2 windows环境

# 安装virtualenv
pip install virtualenv
# 创建虚拟机环境
python -m venv superset-env
# 升级pip
python -m pip install --upgrade pip
# 安装 Superset
pip install apache-superset
# 
pip install Pillow
pip install pymysql
#
pip install MySQLdb
#
set FLASK_APP=superset
# 内容见下面
vi superset_config.py
# 初始化数据库
superset db upgrade
# 创建管理用户
superset fab create-admin
#
superset load-examples
superset init
superset run -p 8088 --with-threads --reload --debugger

superset_config.py的内容如下：

import os
import pymysql
SECRET_KEY = 'WeTzvdirVlm8p5zGVGDjEJWO2jnSkT8b1cybtf3DRPgldYsB6Hjd5sCf'pymysql.install_as_MySQLdb()

superset 提示 Error: No module named ‘MySQLdb’,使用上面的配置就可以了。

1.3 centos环境

Python学习笔记（一）：CentOS 7.9安装配置pyenv、pip、ipython和jupyt
安装pyenv，git无法直接访问，可以离线安装，需要装好多个。

useradd python
su - python
c[python@worker4 ~]$ cat ~/.bash_profile
# .bash_profile# Get the aliases and functions
if [ -f ~/.bashrc ]; then. ~/.bashrc
fi# User specific environment and startup programsPATH=$PATH:$HOME/.local/bin:$HOME/binexport PATHexport PATH="/home/python/.pyenv/bin:$PATH"
eval "$(pyenv init -)"
eval "$(pyenv virtualenv-init -)"source ~/.bash_profile

解压命令不写，按照下面的目录 结构来。


离线安装python

mkdir -pv ~/.pyenv/cache
cd ~/.pyenv/cache
pyenv install 3.8.19
pyenv rehash

su - python
mkdir -p ~/supserset
cd supserset
pyenv local 3.8.19
pyenv virtualenv 3.8.19 superset-env
pyenv activate superset-env
mkdir -pv ~/.pip

安装python3.10，对openssl版本要求在1.1.1，而centos中只有Package 1:openssl-devel-1.0.2k-26.el7_9.x86_64 already installed and latest version

# 升级openssl
# 安装Perl模块管理工具
wget -O- https://cpanmin.us | perl - --sudo App::cpanminus

提示错误，centos7.9上安装python3.10直接放弃。

[root@worker4 openssl-3.0.16]# ./config --prefix=/usr/local/openssl
Can't locate IPC/Cmd.pm in @INC (@INC contains: /home/download/openssl-3.0.16/util/perl /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 . /home/download/openssl-3.0.16/external/perl/Text-Template-1.56/lib) at /home/download/openssl-3.0.16/util/perl/OpenSSL/config.pm line 19.
BEGIN failed--compilation aborted at /home/download/openssl-3.0.16/util/perl/OpenSSL/config.pm line 19.
Compilation failed in require at /home/download/openssl-3.0.16/Configure line 23.
BEGIN failed--compilation aborted at /home/download/openssl-3.0.16/Configure line 23.

于是在rockylinux9.5上部署

-rw-r--r-- 1 python python 19610392 Feb 20 20:59 Python-3.10.16.tar.xz
[python@superset cache]$ pyenv install 3.10.16
Installing Python-3.10.16...
Traceback (most recent call last):File "<string>", line 1, in <module>File "/home/python/.pyenv/versions/3.10.16/lib/python3.10/bz2.py", line 17, in <module>from _bz2 import BZ2Compressor, BZ2Decompressor
ModuleNotFoundError: No module named '_bz2'
WARNING: The Python bz2 extension was not compiled. Missing the bzip2 lib?
Traceback (most recent call last):File "<string>", line 1, in <module>File "/home/python/.pyenv/versions/3.10.16/lib/python3.10/curses/__init__.py", line 13, in <module>from _curses import *
ModuleNotFoundError: No module named '_curses'
WARNING: The Python curses extension was not compiled. Missing the ncurses lib?
Traceback (most recent call last):File "<string>", line 1, in <module>File "/home/python/.pyenv/versions/3.10.16/lib/python3.10/ctypes/__init__.py", line 8, in <module>from _ctypes import Union, Structure, Array
ModuleNotFoundError: No module named '_ctypes'
WARNING: The Python ctypes extension was not compiled. Missing the libffi lib?
Traceback (most recent call last):File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'readline'
WARNING: The Python readline extension was not compiled. Missing the GNU readline lib?
Traceback (most recent call last):File "<string>", line 1, in <module>File "/home/python/.pyenv/versions/3.10.16/lib/python3.10/sqlite3/__init__.py", line 57, in <module>from sqlite3.dbapi2 import *File "/home/python/.pyenv/versions/3.10.16/lib/python3.10/sqlite3/dbapi2.py", line 27, in <module>from _sqlite3 import *
ModuleNotFoundError: No module named '_sqlite3'
WARNING: The Python sqlite3 extension was not compiled. Missing the SQLite3 lib?
Installed Python-3.10.16 to /home/python/.pyenv/versions/3.10.16

sudo yum groupinstall "Development Tools"
sudo yum install -y openssl-devel bzip2-devel libffi-devel \
zlib-devel readline-devel sqlite-devel wget curl

sudo yum install mariadb-connector-c-devel
su - python
cd /home/python/.pyenv/cache
pyenv install 3.10.16
pyenv rehash
mkdir -pv ~/.pyenv/cache
[python@worker4 .pip]$ cat pip.conf 
[global]
index-url=https://mirrors.aliyun.com/pypi/simple/
[install]
trusted-host=mirrors.aliyun.com
[python@worker4 .pip]$ pwd
/home/python/.pipcd ~
mkdir superset
cd superset/
pyenv local 3.10.16
pyenv virtualenv 3.10.16 superset-env
pyenv activate superset-env
pip install apache-superset
pip install pymysql
pip install pydoris
pip install marshmallow_enum
pip install Pillow
export FLASK_APP=superset
superset fab create-admin
superset init
# 开发环境
superset run --host 0.0.0.0 -p 8088 --with-threads --reload --debugger
# 生产环境
gunicorn --workers 4 --bind 0.0.0.0:8088 --timeout 120 --limit-request-line 0 "superset.app:create_app()"

跟猜测的一样，使用python3.10之后，默认安装的就是apache-superset4.1.1

配置脚本

(superset-env) [python@superset superset]$ cat superset_config.py 
import os
import pymysql
SECRET_KEY = '随机的密码'
#
SQLALCHEMY_DATABASE_URI='mysql+pymysql://test:密码@10.101.1.11/superset'
SQLALCHEMY_TRACK_MODIFICATIONS = False
BASE_DIR = os.path.abspath(os.path.dirname(__file__))
STATIC_FOLDER = os.path.join(BASE_DIR, 'static')

防火墙

sudo firewall-cmd --zone=public --add-port=8088/tcp --permanent
sudo firewall-cmd --reload

开机启动

[root@superset shims]# cat /etc/systemd/system/superset.service 
[Unit]
Description=Gunicorn instance to serve Superset
After=network.target[Service]
User=python
Group=python
WorkingDirectory=/home/python/superset
Environment="PATH=/home/python/.pyenv/versions/3.10.16/bin"
ExecStart=/home/python/.pyenv/versions/superset-env/bin/gunicorn --workers 4 --bind 0.0.0.0:8088 --timeout 120 --limit-request-line 0 "superset.app:create_app()"[Install]
WantedBy=multi-user.target