unleashed-firmware/scripts/flipper/assets/tarball.py

42 lines
1 KiB
Python
Raw Normal View History

Updater: resource compression (#3716) * toolbox: compress: moved decompressor implementation to separate func * toolbox: compress: callback-based api; cli: storage unpack command * toolbox: compress: separate r/w contexts for stream api * targets: f18: sync API * compress: naming fixes & cleanup * toolbox: compress: using hs buffer size for stream buffers * toolbox: tar: heatshrink stream mode * toolbox: compress: docs & small cleanup * toolbox: tar: header support for .hs; updater: now uses .hs for resources; .hs.tar: now rewindable * toolbox: compress: fixed hs stream tail handling * updater: reworked progress for resources cleanup; rebalanced stage weights * updater: single-pass decompression; scripts: print resources compression ratio * updater: fixed warnings * toolbox: tar: doxygen * docs: update * docs: info or tarhs format; scripts: added standalone compression/decompression tool for heatshrink-formatted streams * scripts: tarhs: fixed parameter handling * cli: storage extract command; toolbox: tar: guess type based on extension * unit_tests: added test for streamed raw hs decompressor `compress_decode_streamed` * unit_tests: compress: added extraction test for .tar.hs * rpc: autodetect compressed archives * scripts: minor cleanup of common parts * scripts: update: now using in-memory intermediate tar stream * scripts: added hs.py wrapper for heatshrink-related ops (single object and directory-as-tar compression) * scripts: naming fixes * Toolbox: export compress_config_heatshrink_default as const symbol * Toolbox: fix various types naming * Toolbox: more of types naming fixes * Toolbox: use size_t in compress io callbacks and structures * UnitTests: update to match new compress API * Toolbox: proper path_extract_extension usage Co-authored-by: あく <alleteam@gmail.com>
2024-06-30 10:38:48 +00:00
import io
import tarfile
import heatshrink2
from .heatshrink_stream import HeatshrinkDataStreamHeader
FLIPPER_TAR_FORMAT = tarfile.USTAR_FORMAT
TAR_HEATSRINK_EXTENSION = ".ths"
def tar_sanitizer_filter(tarinfo: tarfile.TarInfo):
tarinfo.gid = tarinfo.uid = 0
tarinfo.mtime = 0
tarinfo.uname = tarinfo.gname = "furippa"
return tarinfo
def compress_tree_tarball(
src_dir, output_name, filter=tar_sanitizer_filter, hs_window=13, hs_lookahead=6
):
plain_tar = io.BytesIO()
with tarfile.open(
fileobj=plain_tar,
mode="w:",
format=FLIPPER_TAR_FORMAT,
) as tarball:
tarball.add(src_dir, arcname="", filter=filter)
plain_tar.seek(0)
src_data = plain_tar.read()
compressed = heatshrink2.compress(
src_data, window_sz2=hs_window, lookahead_sz2=hs_lookahead
)
header = HeatshrinkDataStreamHeader(hs_window, hs_lookahead)
with open(output_name, "wb") as f:
f.write(header.pack())
f.write(compressed)
return len(src_data), len(compressed)