The code for handing file overwrite incorrectly assumed that the file on
disk is always contiguous. This resulted in corrupting disk structure
every time when write to existing fragmented file happened. Fix this
by adding proper check for cluster discontinuity and adjust chunk size
on each partial write.
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
This patch partially fixes the issue revealed by the following test
script:
--->8-fat_test1.sh---
#!/bin/bash
make sandbox_defconfig
make
dd if=/dev/zero of=/tmp/10M.img bs=1024 count=10k
mkfs.vfat -v /tmp/10M.img
cat >/tmp/cmds <<EOF
x
host bind 0 /tmp/10M.img
fatls host 0
mw 0x1000000 0x0a434241 0x1000 # "ABC\n"
mw 0x1100000 0x0a464544 0x8000 # "DEF\n"
fatwrite host 0 0x1000000 file0001.raw 0x1000
fatwrite host 0 0x1000000 file0002.raw 0x1000
fatwrite host 0 0x1000000 file0003.raw 0x1000
fatwrite host 0 0x1000000 file0004.raw 0x1000
fatwrite host 0 0x1000000 file0005.raw 0x1000
fatrm host 0 file0002.raw
fatrm host 0 file0004.raw
fatls host 0
fatwrite host 0 0x1100000 file0007.raw 0x4000
fatwrite host 0 0x1100000 file0007.raw 0x4000
reset
EOF
./u-boot </tmp/cmds
#verify
rm -r /tmp/result /tmp/model
mkdir /tmp/result
mkdir /tmp/model
yes ABC | head -c 4096 >/tmp/model/file0001.raw
yes ABC | head -c 4096 >/tmp/model/file0003.raw
yes ABC | head -c 4096 >/tmp/model/file0005.raw
yes DEF | head -c 16384 >/tmp/model/file0007.raw
mcopy -n -i /tmp/10M.img ::file0001.raw /tmp/result
mcopy -n -i /tmp/10M.img ::file0003.raw /tmp/result
mcopy -n -i /tmp/10M.img ::file0005.raw /tmp/result
mcopy -n -i /tmp/10M.img ::file0007.raw /tmp/result
hd /tmp/10M.img
if diff -urq /tmp/model /tmp/result
then
echo Test okay
else
echo Test fail
fi
--->8---
Overwritting a discontiguous test file (file0007.raw) no longer causes
corruption to file0003.raw, which's data lies between the chunks of the
test file. The amount of data written to disk is still incorrect, what
causes damage to the file (file0005.raw), which's data lies next to the
test file. This will be fixed by the next patch.
Feel free to prepare a proper sandbox/py_test based tests based on the
provided test scripts.
linux_compat.c is the best place for kmemdup(), which is currenly used
only in ubifs.c, but will also be used when other kernel files
(in my case, lib/crypto/x509_cert_parser.c and pkcs7_parser.c) will be
imported. So just move it.
Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org>
Reviewed-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
Unlink test for FAT file system seems to fail at test_unlink2.
(When I added this test, I haven't seen any errors though.)
for example,
===8<===
fs_obj_unlink = ['fat', '/home/akashi/tmp/uboot_sandbox_test/128MB.fat32.img']
def test_unlink2(self, u_boot_console, fs_obj_unlink):
"""
Test Case 2 - delete many files
"""
fs_type,fs_img = fs_obj_unlink
with u_boot_console.log.section('Test Case 2 - unlink (many)'):
output = u_boot_console.run_command('host bind 0 %s' % fs_img)
for i in range(0, 20):
output = u_boot_console.run_command_list([
'%srm host 0:0 dir2/0123456789abcdef%02x' % (fs_type, i),
'%sls host 0:0 dir2/0123456789abcdef%02x' % (fs_type, i)])
assert('' == ''.join(output))
output = u_boot_console.run_command(
'%sls host 0:0 dir2' % fs_type)
> assert('0 file(s), 2 dir(s)' in output)
E AssertionError: assert '0 file(s), 2 dir(s)' in ' ./\r\r\n ../\r\r\n 0 0123456789abcdef11\r\r\n\r\r\n1 file(s), 2 dir(s)'
test/py/tests/test_fs/test_unlink.py:52: AssertionError
===>8===
This can happen when fat_itr_next() wrongly detects an already-
deleted directory entry.
File deletion, which was added in the commit f8240ce95d ("fs: fat:
support unlink"), is implemented by marking its entry for a short name
with DELETED_FLAG, but related entry slots for a long file name are kept
unmodified. (So entries will never be actually deleted from media.)
To handle this case correctly, an additional check for a directory slot
will be needed in fat_itr_next().
In addition, I added extra comments about long file name and short file
name format in FAT file system. Although they are not directly related
to the issue, I hope it will be helpful for better understandings
in general.
Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org>
These don't need to be in common.h so move them out into a new header.
Also add some missing comments.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
Drop inclusion of crc.h in common.h and use the correct header directly
instead.
With this we can drop the conflicting definition in fw_env.h and rely on
the crc.h header, which is already included.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
This function is a variant of fs_get_type_name() and returns a filesystem
type with which the current device is associated.
We don't want to export fs_type variable directly because we have to take
care of it consistently within fs.c.
Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org>
Reviewed-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
fs_ls(), fs_mkdir() and fs_unlink() sets fs_type to FS_TYPE_ANY
explicitly, but it is redundant as they call fs_close().
So just remove those lines.
Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org>
Reviewed-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
fs_close() closes the connection to a file system which opened with
either fs_set_blk_dev() or fs_set_dev_with_part(). Many file system
functions implicitly call fs_close(), e.g. fs_closedir(), fs_exist(),
fs_ln(), fs_ls(), fs_mkdir(), fs_read(), fs_size(), fs_write()
and fs_unlink().
So just export it.
Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org>
Reviewed-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
When hitting an invalid FAT cluster while reading a file always print an
error message and return an error code.
Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
File was found on specified location. Info about file was read,
but then immediately destroyed using 'free' call. As a result
file size was set to 0, hence fat process didn't read any data.
Premature 'free' call removed. Resources are freed right before
function return. File is read correctly.
Signed-off-by: Martin Vystrcil <martin.vystrcil@m-linux.cz>
Rename some camel-case variables to match U-Boot style.
Camel case is not generally allowed in U-Boot. Rename this variable to fit
in with the style.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Tested-by: Bin Meng <bmeng.cn@gmail.com>
Sometimes an image has multiple CBFS. The current CBFS API is limited to
handling only one at time. Also it keeps track of the CBFS internally in
BSS, which does not work before relocation, for example.
Add a few new functions to overcome these limitations.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Tested-by: Bin Meng <bmeng.cn@gmail.com>
Move the result variable into the struct also, so that it can be used when
BSS is not available. Add a function to read it.
Note that all functions sill use the BSS version of the data.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Tested-by: Bin Meng <bmeng.cn@gmail.com>
At present there are a number of static variables in BSS. This cannot work
with SPL, at least until BSS is available in board_init_r().
Move the variables into a struct, so it is possible to malloc() it and use
it before BSS is available.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Tested-by: Bin Meng <bmeng.cn@gmail.com>
At present this file has a function at the top, above declarations. This
is normally avoided, so fix it.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Tested-by: Bin Meng <bmeng.cn@gmail.com>
Add a new Kconfig option to enable CBFS in SPL. This can be useful when
the memory-init code is in CBFS.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Tested-by: Bin Meng <bmeng.cn@gmail.com>
Move env_set_hex() over to the new header file along with env_set_addr()
which uses it.
Signed-off-by: Simon Glass <sjg@chromium.org>
Acked-by: Joe Hershberger <joe.hershberger@ni.com>
As part of the effort to remove things from common.h, create a new header
for the gzip functions. Move the function declarations to it and add
missing documentation.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Simon Goldschmidt <simon.k.r.goldschmidt@gmail.com>
In ext4fs_read_file in ext4fs.c, a memset can overwrite the bounds of
the destination memory region. This patch adds a check to disallow
this.
Signed-off-by: Paul Emge <paulemge@forallsecure.com>
This patch checks for 0 in several ext4 headers and gracefully
fails instead of raising a divide-by-0 exception.
Signed-off-by: Paul Emge <paulemge@forallsecure.com>
in ext4fs_read_file, it is possible for a broken/malicious file
system to cause a memcpy of a negative number of bytes, which
overflows all memory. This patch fixes the issue by checking for
a negative length.
Signed-off-by: Paul Emge <paulemge@forallsecure.com>
ext_cache_read doesn't null cache->buf, after freeing, which results
in a later function double-freeing it. This patch fixes
ext_cache_read to call ext_cache_fini instead of free.
Signed-off-by: Paul Emge <paulemge@forallsecure.com>
JOURNAL is optional for EXT4 (and EXT3) filesystems, so add support for
skipping it. This fixes corrupting EXT4 volumes without JOURNAL after
using uboot's 'ext4write' command.
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Reviewed-by: Lukasz Majewski <lukma@denx.de>
The block count entry in the EXT4 filesystem disk structures uses
standard 512-bytes units for most of the typical files. The only
exception are HUGE files, which use the filesystem block size, but those
are not supported by uboot's EXT4 implementation anyway. This patch fixes
the EXT4 code to use proper unit count for inode block count. This fixes
errors reported by fsck.ext4 on disks with non-standard (i.e. 4KiB, in
case of new flash drives) PHYSICAL block size after using 'ext4write'
uboot's command.
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Reviewed-by: Lukasz Majewski <lukma@denx.de>
fatload command can be used to load the EFI payload since EFI system
partition is always a FAT partition. Call into EFI code from do_load()
to set the device path from which the last binary was loaded. An EFI
application like grub2 can’t find its configuration file without the
device path set.
Since device path is now set in do_load() there is no need to set it
in do_load_wrapper() for the load command.
Signed-off-by: Mian Yousaf Kaukab <ykaukab@suse.de>
Reviewed-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
Contrary to fat12/16, fat32 can have root directory at any location
and its size can be expanded.
Without this patch, root directory won't grow properly and so we will
eventually fail to add files under root directory. Please note that this
can happen even if you delete many files as deleted directory entries
are not reclaimed but just marked as "deleted" under the current
implementation.
Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org>
Tested-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
When a long name directory entry is created, multiple directory entries
may be occupied across a directory cluster boundary. Since only one
directory cluster is cached in a directory iterator, a first cluster must
be written back to device before switching over a second cluster.
Without this patch, some added files may be lost even if you don't see
any failures on write operation.
Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org>
Tested-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
With the commit below, fat now correctly handles a file read under
a non-cluster-aligned root directory of fat12/16.
Write operation should be fixed in the same manner.
Fixes: commit 9b18358dc0 ("fs: fat: fix reading non-cluster-aligned
root directory")
Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org>
Cc: Anssi Hannula <anssi.hannula@bitwise.fi>
Tested-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
fat_itr_root() allocates fatbuf so we free it on the exit path, if
the function fails we should not free it, check the return value
and skip freeing if the function fails.
Signed-off-by: Andrew F. Davis <afd@ti.com>
This adds decompression support for Zstandard, which has been included
in Linux btrfs driver for some time.
Signed-off-by: Marek Behún <marek.behun@nic.cz>
The btrfs implementation methods .ls(), .size() and .read() returns 1 on
failure, but the command handlers expect values <0 on failure.
For example if given a nonexistent path, the load command currently
returns success, and hush scripting does not work.
Fix this by setting return values of these methods to -1 instead of 1 on
failure.
Signed-off-by: Marek Behún <marek.behun@nic.cz>
Per Pierre this change shouldn't have been applied as it was superseded
by "fs: btrfs: fix btrfs_search_tree invalid results" which is also
applied now as 1627e5e598.
This reverts commit 633967f981.
Signed-off-by: Tom Rini <trini@konsulko.com>
btrfs_search_tree should return the first item in the tree that is
greater or equal to the searched item.
The search algorithm did not properly handle the edge case where the
searched item is higher than the last item of the node but lower than
the first item of the next node. Instead of properly returning the first
item of the next node, it was returning an invalid path pointer
(pointing to a non-existent item after the last item of the node + 1).
This fixes two issues in the btrfs driver:
- Looking for a ROOT_ITEM could fail if it was the first item of its
leaf node.
- Iterating through DIR_INDEX entries (for readdir) could fail if the
first DIR_INDEX entry was the first item of a leaf node.
Signed-off-by: Pierre Bourdon <delroth@gmail.com>
Cc: Marek Behun <marek.behun@nic.cz>
ROOT_ITEMs in btrfs are referenced without knowing their actual "offset"
value. To perform these searches using only two items from the key, the
btrfs driver uses a special "btrfs_search_tree_key_type" function.
The algorithm used by that function to transform a 3-tuple search into a
2-tuple search was subtly broken, leading to items not being found if
they were the first in their tree node.
This commit fixes btrfs_search_tree_key_type to properly behave in these
situations.
Signed-off-by: Pierre Bourdon <delroth@gmail.com>
Cc: Marek Behun <marek.behun@nic.cz>
Ext4 allows for arbitrarily sized block group descriptors when 64-bit
addressing is enabled, which was previously not properly supported. This
patch dynamically allocates a chunk of memory of the correct size.
Signed-off-by: Benjamin Lim <jarsp.ctf@gmail.com>
A FAT12/FAT16 root directory location is specified by a sector offset and
it might not start at a cluster boundary. It also resides before the
data area (before cluster 2).
However, the current code assumes that the root directory is located at
a beginning of a cluster, causing no files to be found if that is not
the case.
Since the FAT12/FAT16 root directory is located before the data area
and is not aligned to clusters, using unsigned cluster numbers to refer
to the root directory does not work well (the "cluster number" may be
negative, and even allowing it be signed would not make it properly
aligned).
Modify the code to not use the normal cluster numbering when referring to
the root directory of FAT12/FAT16 and instead use a cluster-sized
offsets counted from the root directory start sector.
This is a relatively common case as at least the filesystem formatter on
Win7 seems to create such filesystems by default on 2GB USB sticks when
"FAT" is selected (cluster size 64 sectors, rootdir size 32 sectors,
rootdir starts at half a cluster before cluster 2).
dosfstools mkfs.vfat does not seem to create affected filesystems.
Signed-off-by: Anssi Hannula <anssi.hannula@bitwise.fi>
Reviewed-by: Bernhard Messerklinger <bernhard.messerklinger@br-automation.com>
Tested-by: Bernhard Messerklinger <bernhard.messerklinger@br-automation.com>
Hi,
when I try to load a sparse file via ext4load, I am getting the error message
'invalid extent'
After a deeper look in the code, it seems to be an issue in the function ext4fs_get_extent_block in fs/ext4/ext4_common.c:
The file starts with 1k of zeros. The blocksize is 1024. So the first extend block contains the following information:
eh_entries: 1
eh_depth: 1
ei_block 1
When the upper layer (ext4fs_read_file) asks for fileblock 0, we are running in the 'invalid extent' error message.
For me it seems, that the code is not prepared for handling a sparse block at the beginning of the file. The following change, solved my problem:
I am really not an expert in ext4 filesystems. Can somebody please have a look at this issue and give me a feedback, if I am totally wrong or not?
The command line is:
ln <interface> <dev[:part]> target linkname
Currently symbolic links are supported only in ext4 and only if the option
CMD_EXT4_WRITE is enabled.
Signed-off-by: Jean-Jacques Hiblot <jjhiblot@ti.com>
Reviewed-by: Tom Rini <trini@konsulko.com>
Re-use the functions used to write/create a file, to support creation of a
symbolic link.
The difference with a regular file are small:
- The inode mode is flagged with S_IFLNK instead of S_IFREG
- The ext2_dirent's filetype is FILETYPE_SYMLINK instead of FILETYPE_REG
- Instead of storing the content of a file in allocated blocks, the path
to the target is stored. And if the target's path is short enough, no block
is allocated and the target's path is stored in ext2_inode.b.symlink
As with regulars files, if a file/symlink with the same name exits, it is
unlinked first and then re-created.
Signed-off-by: Jean-Jacques Hiblot <jjhiblot@ti.com>
Reviewed-by: Tom Rini <trini@konsulko.com>
[trini: Fix ext4 env code]
Signed-off-by: Tom Rini <trini@konsulko.com>
There is no need to modify the buffer passed to ext4fs_write_file().
The memset() call is not required here and was likely copied from the
equivalent part of the ext4fs_read_file() function where we do need it.
Signed-off-by: Jean-Jacques Hiblot <jjhiblot@ti.com>
Reviewed-by: Tom Rini <trini@konsulko.com>
When a file contains extents, U-Boot currently reads extent-related data
for each block in the file, even if that data is located in the same
block each time. This significantly slows down loading of files that use
extents. Implement a very dumb cache to prevent repeatedly reading the
same block. Files with extents now load as fast as files without.
Note: There are many cases where read_allocated_block() is called. This
patch only addresses one of those places; all others still read redundant
data in any case they did before. This is a minimal patch to fix the
load command; other cases aren't fixed.
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Before printk.h was introduced and MTDDEBUG was removed,
pr_crit() was calling MTDDEBUG(), which was since then
replaced by the current pr_debug().
pr_debug is more appropriate here.
Signed-off-by: Eran Matityahu <eran.m@variscite.com>
Reviewed-by: Heiko Schocher <hs@denx.de>
U-Boot doesn't support metadata_csum feature. Writing to filesystem with
metadata_csum feature makes the filesystem corrupted and unbootable by
Linux:
[ 2.527495] EXT4-fs (mmcblk0p2): ext4_check_descriptors: Checksum for group 0 failed (52188!=0)
[ 2.537421] EXT4-fs (mmcblk0p2): ext4_check_descriptors: Checksum for group 1 failed (5262!=0)
...
[ 2.653308] EXT4-fs (mmcblk0p2): ext4_check_descriptors: Checksum for group 14 failed (42611!=0)
[ 2.662179] EXT4-fs (mmcblk0p2): ext4_check_descriptors: Checksum for group 15 failed (21527!=0)
[ 2.687920] JBD2: journal checksum error
[ 2.691982] EXT4-fs (mmcblk0p2): error loading journal
[ 2.698292] VFS: Cannot open root device "mmcblk0p2" or unknown-block(179,2): error -74
Don't write to filesystem with meatadata_csum feature to not corrupt the
filesystem.
Signed-off-by: Sébastien Szymanski <sebastien.szymanski@armadeus.com>
