【方案分享】第十一届 “中国软件杯”大学生软件设计大赛遥感解译赛道 比赛方案分享

RandomDistort

resnet18-->hrnet18

more steplr = 0.0004adamw optimizer 
RandomCrop 384RandomFlipOrRotationRandoBlurRandomSwap

flip预测
缩放logit值
形态学后处理

# 清除cell输出结果def clear_output():
    """
    clear output for both jupyter notebook and the console
    """
    import os
    os.system('cls' if os.name == 'nt' else 'clear')    from IPython.display import clear_output as clear
    clear()

# 解压数据!unzip -oq /home/aistudio/data/data134796/train_data.zip -d /home/aistudio/data/src/
!unzip -oq /home/aistudio/data/data134796/test_data.zip -d /home/aistudio/data/src/

# 安装依赖库!pip install -r /home/aistudio/PaddleCD/requirements.txt
clear_output()

# 将0、255修改为0、1!python /home/aistudio/work/data_generate.py /home/aistudio/data/src/train/label# 直接拷贝已有的数据集划分文件，保证每次训练评价指标相同!cp /home/aistudio/work/val.txt /home/aistudio/data/src/val.txt# !cp /home/aistudio/work/train.txt /home/aistudio/data/src/train.txt!cp /home/aistudio/work/train8cp.txt /home/aistudio/data/src/train.txt # 将训练集复制8次，减少每次读取训练集的损耗时间

# 数据统计# !python /home/aistudio/work/data_sta.py

# 配置文件路径与模型保存路径# COF_PATH = '/home/aistudio/PaddleCD/bit_config/bit_baseline.yml'# MD_PATH = '/home/aistudio/best_model/bit_baseline.pdparams'# COF_PATH = '/home/aistudio/PaddleCD/bit_config/bit_res18.yml'# MD_PATH = '/home/aistudio/best_model/bit_res18.pdparams'COF_PATH = '/home/aistudio/PaddleCD/bit_config/bit_hr18.yml'MD_PATH = '/home/aistudio/best_model/bit_hr18.pdparams'

# 模型训练print(COF_PATH.center(100, '*'),'\n\n')print(MD_PATH.center(100, '*'),'\n\n')# 与paddlers训练轮数相同，训练时长55min!python /home/aistudio/PaddleCD/train.py \
       --save_dir /home/aistudio/data/output \
       --config $COF_PATH \
       --do_eval \
       --use_vdl \
       --iters 3700 \
       --save_interval 370 \
       --batch_size=16 \
       --log_iters 50 \
       --num_workers 8 \
       --seed 1919810
       # 后台训练时，需要清空输出，保证notebook正常导入；notebook训练时，可以不用# clear_output()

# 将最优模型拷贝到/home/aistudio/best_model路径下，将训练日志拷贝到/home/aistudio/model_log路径下!mkdir /home/aistudio/best_model/
!cp /home/aistudio/data/output/best_model/model.pdparams $MD_PATH
!find /home/aistudio/data/output -name '*.log' -exec cp -t /home/aistudio/model_log/ "{}" +

# 普通测试!python /home/aistudio/PaddleCD/val.py \
       --batch_size 1 \
       --config $COF_PATH \
       --model_path $MD_PATH

# 增强测试，加入翻转预测!python /home/aistudio/PaddleCD/val.py \
       --aug_eval \
       --flip_vertical \
       --batch_size 1 \
       --config $COF_PATH \
       --model_path $MD_PATH

# # 增强测试，加入多尺度预测# !python /home/aistudio/PaddleCD/val.py \#        --aug_eval \#        --flip_vertical \#        --batch_size 1 \#        --scales 0.5 1.0 2.0\#        --config $COF_PATH \#        --model_path $MD_PATH

# # 增强测试，加入交换预测# !python /home/aistudio/PaddleCD/val.py \#        --aug_eval \#        --swap 7\#        --flip_vertical \#        --batch_size 1 \#        --scales 0.5 1.0 2.0\#        --config $COF_PATH \#        --model_path $MD_PATH

# 普通推理，推理结果保存在/home/aistudio/data/src/result下# bit_hr18推理时长为1分35秒!python /home/aistudio/PaddleCD/predict.py \
       --config $COF_PATH \
       --model_path $MD_PATH \
       --image_path /home/aistudio/data/src/test/A \
       --image_path2 /home/aistudio/data/src/test/B \
       --batch_size 32 \
       --save_dir /home/aistudio/data/src

# 增强推理，加入翻转预测，推理结果保存在/home/aistudio/data/src/result下# bit_hr18推理时长为2分46秒!python /home/aistudio/PaddleCD/predict.py \
       --config $COF_PATH \
       --model_path $MD_PATH \
       --image_path /home/aistudio/data/src/test/A \
       --image_path2 /home/aistudio/data/src/test/B \
       --aug_pred \
       --flip_vertical \
       --batch_size 1 \
       --save_dir /home/aistudio/data/src

# # 增强推理，加入多尺度预测，推理结果保存在/home/aistudio/data/src/result下# !python /home/aistudio/PaddleCD/predict.py \#        --config $COF_PATH \#        --model_path $MD_PATH \#        --image_path /home/aistudio/data/src/test/A \#        --image_path2 /home/aistudio/data/src/test/B \#        --aug_pred \#        --flip_vertical \#        --batch_size 1 \#        --scales 0.5 1.0 2.0 \#        --save_dir /home/aistudio/data/src

# # 增强推理，加入交换预测推理结果保存在/home/aistudio/data/src/result下# !python /home/aistudio/PaddleCD/predict.py \#        --config $COF_PATH \#        --model_path $MD_PATH \#        --image_path /home/aistudio/data/src/test/A \#        --image_path2 /home/aistudio/data/src/test/B \#        --aug_pred \#        --swap 7 \#        --flip_vertical \#        --batch_size 1 \#        --scales 0.5 1.0 2.0 \#        --save_dir /home/aistudio/data/src

# # 滑窗增强推理，推理结果保存在/home/aistudio/data/src/result下# bit_hr18推理时长为12分# !python /home/aistudio/PaddleCD/predict.py \#        --config $COF_PATH \#        --model_path $MD_PATH \#        --image_path /home/aistudio/data/src/test/A \#        --image_path2 /home/aistudio/data/src/test/B \#        --aug_pred \#        --flip_vertical \#        --is_slide \#        --batch_size 1 \#        --patch_size 16 \#        --crop_size 384 384 \#        --stride 128 128 \#        --save_dir /home/aistudio/data/src

# 将0、1变为0、255!python /home/aistudio/work/data_postprocess.py

# 结果打包!zip -j /home/aistudio/submisson.zip /home/aistudio/data/src/result/* > /dev/null

SystemError: (Fatal) DataLoader process (pid   1. If run DataLoader by DataLoader.from_generator(...), queue capacity is set by from_generator(..., capacity=xx, ...).  2. If run DataLoader by DataLoader(dataset, ...), queue capacity is set as 2 times of the max value of num_workers and len(places).  3. If run by DataLoader(dataset, ..., use_shared_memory=True), set use_shared_memory=False for not using shared memory.) exited is killed by signal: 2992.
  It may be caused by insufficient shared storage space. This problem usually occurs when using docker as a development environment.
  Please use command `df -h` to check the storage space of `/dev/shm`. Shared storage space needs to be greater than (DataLoader Num * DataLoader queue capacity * 1 batch data size).
  You can solve this problem by increasing the shared storage space or reducing the queue capacity appropriately.
Bus error (at /paddle/paddle/fluid/imperative/data_loader.cc:177)

                   bg     changePixel_sum   636876269   31066643Pixel_pct    0.953489  0.0465109Sample_sum        638        573Sample_pct   0.526837   0.473163