cluster-commits April 2017

cluster-commits@lists.fedorahosted.org

1 participants
47 discussions

[gfs2-utils] 22/38: fsck.gfs2: Repair rindex entries that were destroyed
by git repository hosting 18 Apr '17

18 Apr '17

This is an automated email from the git hooks/post-receive script. andyp pushed a commit to branch RHEL7 in repository gfs2-utils. commit c0fa9f938adb40eb1b52612416c03c37bd1d2b35 Author: Bob Peterson <rpeterso(a)redhat.com> Date: Fri Jan 13 14:42:14 2017 -0500 fsck.gfs2: Repair rindex entries that were destroyed If an rindex entry was destroyed, the rindex was skipped, which means its rgrp never gets inserted into the rgrp tree. The trouble is, that causes a off-by-one-entry problem with all subsequent rindex entries. So if rindex entry R is destroyed, then rebuilt, originally good entry R+1 is checked against new entry for R. Function rg_repair already had the capability to insert new rindex entries, but only at the end. This would be typical for a gfs2_grow that failed and left a bunch of rgrps on the device, but the rindex was never synced properly. This patch allows the function to insert catch-up entries in the middle, which re-syncs the actual versus expected rgrp trees. Signed-off-by: Bob Peterson <rpeterso(a)redhat.com> --- gfs2/fsck/rgrepair.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/gfs2/fsck/rgrepair.c b/gfs2/fsck/rgrepair.c index 49bbad6..f397db1 100644 --- a/gfs2/fsck/rgrepair.c +++ b/gfs2/fsck/rgrepair.c @@ -1137,10 +1137,14 @@ int rg_repair(struct gfs2_sbd *sdp, int trust_lvl, int *rg_count, int *sane) next = osi_next(n); enext = osi_next(e); expected = (struct rgrp_tree *)e; + actual = (struct rgrp_tree *)n; - /* If we ran out of actual rindex entries due to rindex + /* If the next "actual" rgrp in memory is too far away, + fill in a new one with the expected value. -or- + If we ran out of actual rindex entries due to rindex damage, fill in a new one with the expected values. */ - if (!n) { /* end of actual rindex */ + if (!n || /* end of actual rindex */ + expected->ri.ri_addr < actual->ri.ri_addr) { log_err( _("Entry missing from rindex: 0x%llx\n"), (unsigned long long)expected->ri.ri_addr); actual = rgrp_insert(&sdp->rgtree, @@ -1150,8 +1154,9 @@ int rg_repair(struct gfs2_sbd *sdp, int trust_lvl, int *rg_count, int *sane) break; } rindex_modified = 1; + next = n; /* Ensure that the old actual gets checked + against a new expected, since we added */ } else { - actual = (struct rgrp_tree *)n; ri_compare(rg, actual->ri, expected->ri, ri_addr, "llx", unsigned long long); ri_compare(rg, actual->ri, expected->ri, ri_length, -- To stop receiving notification emails like this one, please contact the administrator of this repository.

1 0

[gfs2-utils] 21/38: fsck.gfs2: allow rindex recovery when too many segments
by git repository hosting 18 Apr '17

18 Apr '17

This is an automated email from the git hooks/post-receive script. andyp pushed a commit to branch RHEL7 in repository gfs2-utils. commit 9aee0d0d382f6ef02b0d35338c2cd7fe3b1f603b Author: Bob Peterson <rpeterso(a)redhat.com> Date: Fri Jan 13 14:36:56 2017 -0500 fsck.gfs2: allow rindex recovery when too many segments Patch 0103dc0 (fsck.gfs2: Detect multiple rgrp grow segments) added an array of rgrp segments. If it found too many segments, it threw up its hands and gave up and refused to try to fix it. But in most cases, proceeding to fix the rgrps won't cause any harm, since the rgrps are checked for integrity anyway. This patch allows fsck to try to fix the rgrp corruption, even after the maximum number of rgrp segments has been reached. Signed-off-by: Bob Peterson <rpeterso(a)redhat.com> --- gfs2/fsck/rgrepair.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/gfs2/fsck/rgrepair.c b/gfs2/fsck/rgrepair.c index 7479458..49bbad6 100644 --- a/gfs2/fsck/rgrepair.c +++ b/gfs2/fsck/rgrepair.c @@ -230,9 +230,8 @@ static int find_shortest_rgdist(struct gfs2_sbd *sdp, uint64_t *dist_array, } if (gsegment >= MAX_RGSEGMENTS) { log_err(_("Maximum number of rgrp grow segments reached.\n")); - log_err(_("This file system cannot be repaired with fsck.\n")); - gsegment = 0; - goto out; + log_err(_("This file system has more than %d resource " + "group segments.\n"), MAX_RGSEGMENTS); } /* -------------------------------------------------------------- */ /* Sanity-check our first_rg_dist. If RG #2 got nuked, the */ @@ -260,7 +259,6 @@ static int find_shortest_rgdist(struct gfs2_sbd *sdp, uint64_t *dist_array, (unsigned long long)*dist_array); } /* if first RG distance is within tolerance */ -out: gfs2_special_free(&false_rgrps); return gsegment; } -- To stop receiving notification emails like this one, please contact the administrator of this repository.

1 0

[gfs2-utils] 20/38: fsck.gfs2: Prevent rgrp segment overflow
by git repository hosting 18 Apr '17

18 Apr '17

This is an automated email from the git hooks/post-receive script. andyp pushed a commit to branch RHEL7 in repository gfs2-utils. commit aee013ddaec1aaa396a894107241e9372a7e6bb2 Author: Bob Peterson <rpeterso(a)redhat.com> Date: Fri Jan 13 14:30:13 2017 -0500 fsck.gfs2: Prevent rgrp segment overflow Patch 0103dc0 (fsck.gfs2: Detect multiple rgrp grow segments) allowed fsck to detect multiple gfs2_grow segments that may have placed resource groups on unexpected boundaries. However, it failed to check or take appropriate action when too many segments were found. Instead, it would write past the arrays and corrupt memory. This patch adds some sanity checks to prevent this from happening. Signed-off-by: Bob Peterson <rpeterso(a)redhat.com> --- gfs2/fsck/rgrepair.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/gfs2/fsck/rgrepair.c b/gfs2/fsck/rgrepair.c index 55057ce..7479458 100644 --- a/gfs2/fsck/rgrepair.c +++ b/gfs2/fsck/rgrepair.c @@ -164,6 +164,8 @@ static int find_shortest_rgdist(struct gfs2_sbd *sdp, uint64_t *dist_array, /* That last one didn't pan out, so: */ dist_cnt[gsegment]--; gsegment++; + if (gsegment >= MAX_RGSEGMENTS) + break; } if ((blk - block_last_rg) > (524288 * 2)) { log_info(_("No rgrps were found within 4GB " @@ -217,6 +219,8 @@ static int find_shortest_rgdist(struct gfs2_sbd *sdp, uint64_t *dist_array, } } else { gsegment++; + if (gsegment >= MAX_RGSEGMENTS) + break; } block_last_rg = blk; if (rgs_sampled < 6) @@ -224,7 +228,7 @@ static int find_shortest_rgdist(struct gfs2_sbd *sdp, uint64_t *dist_array, else blk += shortest_dist_btwn_rgs - 1; } - if (gsegment > MAX_RGSEGMENTS) { + if (gsegment >= MAX_RGSEGMENTS) { log_err(_("Maximum number of rgrp grow segments reached.\n")); log_err(_("This file system cannot be repaired with fsck.\n")); gsegment = 0; -- To stop receiving notification emails like this one, please contact the administrator of this repository.

1 0

[gfs2-utils] 19/38: libgfs2: Make rg repair do better sanity checks on rindex
by git repository hosting 18 Apr '17

18 Apr '17

This is an automated email from the git hooks/post-receive script. andyp pushed a commit to branch RHEL7 in repository gfs2-utils. commit c33505e2df033ac5a15e61f8fae95779ce5cf971 Author: Bob Peterson <rpeterso(a)redhat.com> Date: Thu Jan 12 16:09:32 2017 -0500 libgfs2: Make rg repair do better sanity checks on rindex This patch changes the checks done on the rindex file. Before, if the rindex addresses weren't perfectly aligned and equally spaced, it considered the rindex not sane. However, in most cases it should be sane. Rather than flag this as a violation, I use it to trigger a read from disk to see if the rindex really DOES point to an rgrp in these cases, and if not, THEN flag the rindex as not sane. I also added some basic checks on the rindex values, such as checking if the number of bitmaps is sane, and if most of the fields have values we'd expect. Signed-off-by: Bob Peterson <rpeterso(a)redhat.com> --- gfs2/libgfs2/super.c | 66 +++++++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 57 insertions(+), 9 deletions(-) diff --git a/gfs2/libgfs2/super.c b/gfs2/libgfs2/super.c index 81834b3..6175203 100644 --- a/gfs2/libgfs2/super.c +++ b/gfs2/libgfs2/super.c @@ -127,6 +127,59 @@ int read_sb(struct gfs2_sbd *sdp) return 0; } +/* rgd_seems_sane - check some general things about the rindex entry + * + * If rg lengths are not consistent, it's not sane (or it's converted from + * gfs1). The first RG will be a different length due to space reserved for + * the superblock, so we can't detect this until we check rgrp 3, when we + * can compare the distance between rgrp 1 and rgrp 2. + * + * Returns: 1 if the rgd seems relatively sane + */ +static int rgd_seems_sane(struct gfs2_sbd *sdp, struct rgrp_tree *rgd) +{ + uint32_t most_bitmaps_possible; + + /* rg length must be at least 1 */ + if (rgd->ri.ri_length == 0) + return 0; + + /* A max rgrp, 2GB, divided into blocksize, divided by blocks/byte + represented in the bitmap, NBBY. Rough approximation only, due to + metadata headers. I'm doing the math this way to avoid overflow. */ + most_bitmaps_possible = (GFS2_MAX_RGSIZE * 1024 * 256) / sdp->bsize; + if (rgd->ri.ri_length > most_bitmaps_possible) + return 0; + + if (rgd->ri.ri_data0 != rgd->ri.ri_addr + rgd->ri.ri_length) + return 0; + + if (rgd->ri.ri_bitbytes != rgd->ri.ri_data / GFS2_NBBY) + return 0; + + return 1; +} + +/* good_on_disk - check if the rindex points to what looks like an rgrp on disk + * + * This is only called when the rindex pointers aren't spaced evenly, which + * isn't often. The rindex is pointing to an unexpected location, so we + * check if the block it is pointing to is really an rgrp. If so, we count the + * rindex entry as "sane" (after all, it did pass the previous checks above.) + * If not, we count it as not sane, and therefore, the whole rindex is not to + * be trusted by fsck.gfs2. + */ +static int good_on_disk(struct gfs2_sbd *sdp, struct rgrp_tree *rgd) +{ + struct gfs2_buffer_head *bh; + int is_rgrp; + + bh = bread(sdp, rgd->ri.ri_addr); + is_rgrp = (gfs2_check_meta(bh, GFS2_METATYPE_RG) == 0); + brelse(bh); + return is_rgrp; +} + /** * rindex_read - read in the rg index file * @sdp: the incore superblock pointer @@ -182,17 +235,12 @@ int rindex_read(struct gfs2_sbd *sdp, int fd, uint64_t *count1, int *sane) if (!sdp->gfs1) { if (prev_rgd->start >= rgd->start) *sane = 0; - /* If rg lengths are not consistent, it's not - sane (or it's converted from gfs1). The - first RG will be a different length due to - space allocated for the superblock, so we - can't detect this until we check rgrp 3, - when we can compare the distance between - rgrp 1 and rgrp 2. */ - if (rg > 2 && prev_length && + else if (!rgd_seems_sane(sdp, rgd)) + *sane = 0; + else if (*sane && rg > 2 && prev_length && prev_length != rgd->start - prev_rgd->start) - *sane = 0; + *sane = good_on_disk(sdp, rgd); } prev_length = rgd->start - prev_rgd->start; prev_rgd->length = rgrp_size(prev_rgd); -- To stop receiving notification emails like this one, please contact the administrator of this repository.

1 0

[gfs2-utils] 18/38: gfs2_edit: Fix unaligned accesses due to saved_metablock size
by git repository hosting 18 Apr '17

18 Apr '17

This is an automated email from the git hooks/post-receive script. andyp pushed a commit to branch RHEL7 in repository gfs2-utils. commit f49411b143528300b6d19f390b1f68ea92716e9b Author: Andrew Price <anprice(a)redhat.com> Date: Thu Nov 10 00:08:44 2016 +0000 gfs2_edit: Fix unaligned accesses due to saved_metablock size Each saved block is prefixed by a saved_metablock structure that holds its block number and data length. Unfortunately this structure is 10 bytes in size, which means that storing the metadata header in memory immediately following it causes accesses to the header fields to be unaligned. This causes testsuite failures on sparc64. Instead, on restore, use a separate buffer for the block data to ensure aligned accesses to those fields. Reported-By: Valentin Vidic <Valentin.Vidic(a)CARNet.hr> Signed-off-by: Andrew Price <anprice(a)redhat.com> --- gfs2/edit/savemeta.c | 59 +++++++++++++++++++++++++--------------------------- 1 file changed, 28 insertions(+), 31 deletions(-) diff --git a/gfs2/edit/savemeta.c b/gfs2/edit/savemeta.c index ac2dd9f..f6b7ff0 100644 --- a/gfs2/edit/savemeta.c +++ b/gfs2/edit/savemeta.c @@ -44,7 +44,6 @@ struct savemeta_header { struct saved_metablock { uint64_t blk; uint16_t siglen; /* significant data length */ - char buf[]; /* This needs to be packed because old versions of gfs2_edit read and write the individual fields separately, so the hole after siglen must be eradicated before the struct reflects what's on disk. */ @@ -435,7 +434,7 @@ static int save_bh(struct metafd *mfd, struct gfs2_buffer_head *savebh, uint64_t } savedata->blk = cpu_to_be64(savebh->b_blocknr); savedata->siglen = cpu_to_be16(blklen); - memcpy(savedata->buf, savebh->b_data, blklen); + memcpy(savedata + 1, savebh->b_data, blklen); if (savemetawrite(mfd, savedata, outsz) != outsz) { fprintf(stderr, "write error: %s from %s:%d: block %lld (0x%llx)\n", @@ -956,7 +955,7 @@ static off_t restore_init(gzFile gzfd, struct savemeta_header *smh) } -static int restore_block(gzFile gzfd, struct saved_metablock *svb, uint16_t maxlen) +static int restore_block(gzFile gzfd, struct saved_metablock *svb, char *buf, uint16_t maxlen) { int gzerr; int ret; @@ -988,8 +987,8 @@ static int restore_block(gzFile gzfd, struct saved_metablock *svb, uint16_t maxl return -1; } - if (maxlen) { - ret = gzread(gzfd, svb + 1, svb->siglen); + if (buf != NULL && maxlen != 0) { + ret = gzread(gzfd, buf, svb->siglen); if (ret < svb->siglen) { goto gzread_err; } @@ -1011,17 +1010,16 @@ gzread_err: static int restore_super(gzFile gzfd, off_t pos) { int ret; - struct saved_metablock *svb; + struct saved_metablock svb = {0}; struct gfs2_buffer_head dummy_bh; - size_t len = sizeof(*svb) + sizeof(struct gfs2_sb); - svb = calloc(1, len); - if (svb == NULL) { + dummy_bh.b_data = calloc(1, sizeof(struct gfs2_sb)); + if (dummy_bh.b_data == NULL) { perror("Failed to restore super block"); exit(1); } gzseek(gzfd, pos, SEEK_SET); - ret = restore_block(gzfd, svb, sizeof(struct gfs2_sb)); + ret = restore_block(gzfd, &svb, dummy_bh.b_data, sizeof(struct gfs2_sb)); if (ret == 1) { fprintf(stderr, "Reached end of file while restoring superblock\n"); goto err; @@ -1029,7 +1027,6 @@ static int restore_super(gzFile gzfd, off_t pos) goto err; } - dummy_bh.b_data = (char *)svb->buf; gfs2_sb_in(&sbd.sd_sb, &dummy_bh); sbd1 = (struct gfs_sb *)&sbd.sd_sb; ret = check_sb(&sbd.sd_sb); @@ -1040,11 +1037,11 @@ static int restore_super(gzFile gzfd, off_t pos) if (ret == 1) sbd.gfs1 = 1; sbd.bsize = sbd.sd_sb.sb_bsize; - free(svb); + free(dummy_bh.b_data); printf("Block size is %uB\n", sbd.bsize); return 0; err: - free(svb); + free(dummy_bh.b_data); return -1; } @@ -1056,7 +1053,7 @@ static int find_highest_block(gzFile gzfd, off_t pos, uint64_t fssize) while (1) { gzseek(gzfd, pos, SEEK_SET); - err = restore_block(gzfd, &svb, 0); + err = restore_block(gzfd, &svb, NULL, 0); if (err == 1) break; if (err != 0) @@ -1081,12 +1078,12 @@ static int find_highest_block(gzFile gzfd, off_t pos, uint64_t fssize) static int restore_data(int fd, gzFile gzin_fd, off_t pos, int printonly) { - struct saved_metablock *savedata; - size_t insz = sizeof(*savedata) + sbd.bsize; + struct saved_metablock savedata = {0}; uint64_t writes = 0; + char *buf; - savedata = calloc(1, insz); - if (savedata == NULL) { + buf = calloc(1, sbd.bsize); + if (buf == NULL) { perror("Failed to restore data"); exit(1); } @@ -1095,36 +1092,36 @@ static int restore_data(int fd, gzFile gzin_fd, off_t pos, int printonly) blks_saved = 0; while (TRUE) { int err; - err = restore_block(gzin_fd, savedata, sbd.bsize); + err = restore_block(gzin_fd, &savedata, buf, sbd.bsize); if (err == 1) break; if (err != 0) { - free(savedata); + free(buf); return -1; } if (printonly) { struct gfs2_buffer_head dummy_bh = { - .b_data = savedata->buf, - .b_blocknr = savedata->blk, + .b_data = buf, + .b_blocknr = savedata.blk, }; - if (printonly > 1 && printonly == savedata->blk) { + if (printonly > 1 && printonly == savedata.blk) { display_block_type(&dummy_bh, TRUE); display_gfs2(&dummy_bh); break; } else if (printonly == 1) { - print_gfs2("%"PRId64" (l=0x%x): ", blks_saved, savedata->siglen); + print_gfs2("%"PRId64" (l=0x%x): ", blks_saved, savedata.siglen); display_block_type(&dummy_bh, TRUE); } } else { - warm_fuzzy_stuff(savedata->blk, FALSE); - memset(savedata->buf + savedata->siglen, 0, sbd.bsize - savedata->siglen); - if (pwrite(fd, savedata->buf, sbd.bsize, savedata->blk * sbd.bsize) != sbd.bsize) { + warm_fuzzy_stuff(savedata.blk, FALSE); + memset(buf + savedata.siglen, 0, sbd.bsize - savedata.siglen); + if (pwrite(fd, buf, sbd.bsize, savedata.blk * sbd.bsize) != sbd.bsize) { fprintf(stderr, "write error: %s from %s:%d: block %lld (0x%llx)\n", strerror(errno), __FUNCTION__, __LINE__, - (unsigned long long)savedata->blk, - (unsigned long long)savedata->blk); - free(savedata); + (unsigned long long)savedata.blk, + (unsigned long long)savedata.blk); + free(buf); return -1; } writes++; @@ -1135,7 +1132,7 @@ static int restore_data(int fd, gzFile gzin_fd, off_t pos, int printonly) } if (!printonly) warm_fuzzy_stuff(sbd.fssize, 1); - free(savedata); + free(buf); return 0; } -- To stop receiving notification emails like this one, please contact the administrator of this repository.

1 0

[gfs2-utils] 17/38: gfs2_edit: Fix unaligned access in restore_init()
by git repository hosting 18 Apr '17

18 Apr '17

This is an automated email from the git hooks/post-receive script. andyp pushed a commit to branch RHEL7 in repository gfs2-utils. commit 0fa27cdd64946c49823669e7ab77d8a06d9a00b9 Author: Andrew Price <anprice(a)redhat.com> Date: Wed Nov 9 18:27:12 2016 +0000 gfs2_edit: Fix unaligned access in restore_init() Scanning this buffer byte-by-byte using casts results in a quick bus error on sparc64. Avoid this by using a memcpy. This function only runs once at the beginning of a restoremeta and in most cases the loop will only execute once so this adds very little overhead. Reported-By: Valentin Vidic <Valentin.Vidic(a)CARNet.hr> Signed-off-by: Andrew Price <anprice(a)redhat.com> --- gfs2/edit/savemeta.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/gfs2/edit/savemeta.c b/gfs2/edit/savemeta.c index c9b51e9..ac2dd9f 100644 --- a/gfs2/edit/savemeta.c +++ b/gfs2/edit/savemeta.c @@ -924,8 +924,7 @@ static off_t restore_init(gzFile gzfd, struct savemeta_header *smh) size_t rs; char buf[256]; off_t startpos = 0; - struct saved_metablock *svb; - struct gfs2_meta_header *sbmh; + struct gfs2_meta_header sbmh; err = read_header(gzfd, smh); if (err < 0) { @@ -943,14 +942,15 @@ static off_t restore_init(gzFile gzfd, struct savemeta_header *smh) exit(1); } /* Scan for the beginning of the file body. Required to support old formats(?). */ - for (i = 0; i < (256 - sizeof(*svb) - sizeof(*sbmh)); i++) { - svb = (struct saved_metablock *)&buf[i]; - sbmh = (struct gfs2_meta_header *)svb->buf; - if (sbmh->mh_magic == cpu_to_be32(GFS2_MAGIC) && - sbmh->mh_type == cpu_to_be32(GFS2_METATYPE_SB)) + for (i = 0; i < (256 - sizeof(struct saved_metablock) - sizeof(sbmh)); i++) { + off_t off = i + sizeof(struct saved_metablock); + + memcpy(&sbmh, &buf[off], sizeof(sbmh)); + if (sbmh.mh_magic == cpu_to_be32(GFS2_MAGIC) && + sbmh.mh_type == cpu_to_be32(GFS2_METATYPE_SB)) break; } - if (i == (sizeof(buf) - sizeof(*svb) - sizeof(*sbmh))) + if (i == (sizeof(buf) - sizeof(struct saved_metablock) - sizeof(sbmh))) i = 0; return startpos + i; /* File offset of saved sb */ } -- To stop receiving notification emails like this one, please contact the administrator of this repository.

1 0

[gfs2-utils] 16/38: gfs2_edit savemeta: Follow lf_next
by git repository hosting 18 Apr '17

18 Apr '17

This is an automated email from the git hooks/post-receive script. andyp pushed a commit to branch RHEL7 in repository gfs2-utils. commit 7dd11ca7f283ce7283100668ec44fc386cc74f21 Author: Andrew Price <anprice(a)redhat.com> Date: Mon Dec 12 22:25:01 2016 +0000 gfs2_edit savemeta: Follow lf_next Previously, gfs2_edit savemeta did not walk down the lf_next chain to save the directory leaf blocks that might be there. Add a save_leaf_chain() function that does that. Related: rhbz#1382087 Signed-off-by: Andrew Price <anprice(a)redhat.com> --- gfs2/edit/savemeta.c | 41 ++++++++++++++++++++++++++++++++--------- 1 file changed, 32 insertions(+), 9 deletions(-) diff --git a/gfs2/edit/savemeta.c b/gfs2/edit/savemeta.c index 9723dcd..c9b51e9 100644 --- a/gfs2/edit/savemeta.c +++ b/gfs2/edit/savemeta.c @@ -539,12 +539,41 @@ static void save_indirect_blocks(struct metafd *mfd, osi_list_t *cur_list, } /* for all data on the indirect block */ } +static int save_leaf_chain(struct metafd *mfd, struct gfs2_sbd *sdp, uint64_t blk) +{ + struct gfs2_buffer_head *bh; + struct gfs2_leaf leaf; + + do { + if (gfs2_check_range(sdp, blk) != 0) + return 0; + bh = bread(sdp, blk); + if (bh == NULL) { + perror("Failed to read leaf block"); + return 1; + } + warm_fuzzy_stuff(blk, FALSE); + if (gfs2_check_meta(bh, GFS2_METATYPE_LF) == 0) { + int ret = save_bh(mfd, bh, blk, NULL); + if (ret != 0) { + brelse(bh); + return ret; + } + } + gfs2_leaf_in(&leaf, bh); + brelse(bh); + blk = leaf.lf_next; + } while (leaf.lf_next != 0); + + return 0; +} + /* * save_inode_data - save off important data associated with an inode * * mfd - destination file descriptor * iblk - block number of the inode to save the data for - * + * * For user files, we don't want anything except all the indirect block * pointers that reside on blocks on all but the highest height. * @@ -626,15 +655,9 @@ static void save_inode_data(struct metafd *mfd, uint64_t iblk) (uint64_t)inode->i_di.di_num.no_addr); exit(-1); } - if (leaf_no == old_leaf || - gfs2_check_range(&sbd, leaf_no) != 0) - continue; + if (leaf_no != old_leaf && save_leaf_chain(mfd, &sbd, leaf_no) != 0) + exit(-1); old_leaf = leaf_no; - mybh = bread(&sbd, leaf_no); - warm_fuzzy_stuff(iblk, FALSE); - if (gfs2_check_meta(mybh, GFS2_METATYPE_LF) == 0) - save_block(sbd.device_fd, mfd, leaf_no, iblk, NULL); - brelse(mybh); } } if (inode->i_di.di_eattr) { /* if this inode has extended attributes */ -- To stop receiving notification emails like this one, please contact the administrator of this repository.

1 0

[gfs2-utils] 15/38: gfs2_edit savemeta: Don't read rgrp blocks twice
by git repository hosting 18 Apr '17

18 Apr '17

This is an automated email from the git hooks/post-receive script. andyp pushed a commit to branch RHEL7 in repository gfs2-utils. commit cf34e461edc0458c61b4d6c070161ba824ec3740 Author: Andrew Price <anprice(a)redhat.com> Date: Fri Feb 12 17:25:38 2016 +0000 gfs2_edit savemeta: Don't read rgrp blocks twice When savemeta iterates over the rgrps and saves the rgrp header blocks it calls save_block which does a bread() for each block despite them already being read in earlier. Instead, call save_bh() on the existing rgrp buffers. This isn't likely to give a significant performance improvement as the duplicate reads would be cache hits but it should have a noticeable effect on very large file systems. Signed-off-by: Andrew Price <anprice(a)redhat.com> --- gfs2/edit/savemeta.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/gfs2/edit/savemeta.c b/gfs2/edit/savemeta.c index 3e8dae0..9723dcd 100644 --- a/gfs2/edit/savemeta.c +++ b/gfs2/edit/savemeta.c @@ -855,8 +855,8 @@ void savemeta(char *out_fn, int saveoption, int gziplevel) } /* Walk through the resource groups saving everything within */ for (n = osi_first(&sbd.rgtree); n; n = osi_next(n)) { - uint64_t blk; struct rgrp_tree *rgd; + unsigned i; rgd = (struct rgrp_tree *)n; if (gfs2_rgrp_read(&sbd, rgd)) @@ -866,10 +866,9 @@ void savemeta(char *out_fn, int saveoption, int gziplevel) (unsigned long long)rgd->ri.ri_addr, rgd->ri.ri_length); /* Save off the rg and bitmaps */ - for (blk = rgd->ri.ri_addr; - blk < rgd->ri.ri_data0; blk++) { - warm_fuzzy_stuff(blk, FALSE); - save_block(sbd.device_fd, &mfd, blk, blk, NULL); + for (i = 0; i < rgd->ri.ri_length; i++) { + warm_fuzzy_stuff(rgd->ri.ri_addr + i, FALSE); + save_bh(&mfd, rgd->bits[i].bi_bh, 0, NULL); } /* Save off the other metadata: inodes, etc. if mode is not 'savergs' */ if (saveoption != 2) { -- To stop receiving notification emails like this one, please contact the administrator of this repository.

1 0

[gfs2-utils] 14/38: gfs2_edit savemeta: Factor out the bh saving code
by git repository hosting 18 Apr '17

18 Apr '17

This is an automated email from the git hooks/post-receive script. andyp pushed a commit to branch RHEL7 in repository gfs2-utils. commit e9b3ac8ed31ae6f3b59e017b21843e246bc88be3 Author: Andrew Price <anprice(a)redhat.com> Date: Fri Feb 12 15:16:36 2016 +0000 gfs2_edit savemeta: Factor out the bh saving code save_block() accepts a block number, reads it and saves it, which makes it tricky to save an already-read block without reading it a second time. Split out the bh saving part of save_block() into save_bh() to make things more flexible. Also switch the return value of save_block to only convey error/success and return the block type through an optional pointer argument instead. Signed-off-by: Andrew Price <anprice(a)redhat.com> --- gfs2/edit/savemeta.c | 74 +++++++++++++++++++++++++++++----------------------- 1 file changed, 41 insertions(+), 33 deletions(-) diff --git a/gfs2/edit/savemeta.c b/gfs2/edit/savemeta.c index db9a947..3e8dae0 100644 --- a/gfs2/edit/savemeta.c +++ b/gfs2/edit/savemeta.c @@ -208,14 +208,16 @@ static int get_gfs_struct_info(struct gfs2_buffer_head *lbh, uint64_t owner, struct gfs2_meta_header mh; struct gfs2_inode *inode; - *block_type = 0; + if (block_type != NULL) + *block_type = 0; *gstruct_len = sbd.bsize; gfs2_meta_header_in(&mh, lbh); if (mh.mh_magic != GFS2_MAGIC) return -1; - *block_type = mh.mh_type; + if (block_type != NULL) + *block_type = mh.mh_type; switch (mh.mh_type) { case GFS2_METATYPE_SB: /* 1 (superblock) */ @@ -405,32 +407,19 @@ static int savemetaclose(struct metafd *mfd) return close(mfd->fd); } -static int save_block(int fd, struct metafd *mfd, uint64_t blk, uint64_t owner) +static int save_bh(struct metafd *mfd, struct gfs2_buffer_head *savebh, uint64_t owner, int *blktype) { - int blktype, blklen; - size_t outsz; - struct gfs2_buffer_head *savebh; struct saved_metablock *savedata; - - if (gfs2_check_range(&sbd, blk) && blk != sbd.sb_addr) { - fprintf(stderr, "\nWarning: bad block pointer '0x%llx' " - "ignored in block (block %llu (0x%llx))", - (unsigned long long)blk, - (unsigned long long)owner, (unsigned long long)owner); - return 0; - } - savebh = bread(&sbd, blk); + size_t outsz; + int blklen; /* If this isn't metadata and isn't a system file, we don't want it. Note that we're checking "owner" here rather than blk. That's because we want to know if the source inode is a system inode not the block within the inode "blk". They may or may not be the same thing. */ - if (get_gfs_struct_info(savebh, owner, &blktype, &blklen) && - !block_is_systemfile(owner)) { - brelse(savebh); + if (get_gfs_struct_info(savebh, owner, blktype, &blklen) && !block_is_systemfile(owner)) return 0; /* Not metadata, and not system file, so skip it */ - } /* No need to save trailing zeroes */ for (; blklen > 0 && savebh->b_data[blklen - 1] == '\0'; blklen--); @@ -444,7 +433,7 @@ static int save_block(int fd, struct metafd *mfd, uint64_t blk, uint64_t owner) perror("Failed to save block"); exit(1); } - savedata->blk = cpu_to_be64(blk); + savedata->blk = cpu_to_be64(savebh->b_blocknr); savedata->siglen = cpu_to_be16(blklen); memcpy(savedata->buf, savebh->b_data, blklen); @@ -459,8 +448,27 @@ static int save_block(int fd, struct metafd *mfd, uint64_t blk, uint64_t owner) blks_saved++; free(savedata); + return 0; +} + +static int save_block(int fd, struct metafd *mfd, uint64_t blk, uint64_t owner, int *blktype) +{ + struct gfs2_buffer_head *savebh; + int err; + + if (gfs2_check_range(&sbd, blk) && blk != sbd.sb_addr) { + fprintf(stderr, "\nWarning: bad block pointer '0x%llx' " + "ignored in block (block %llu (0x%llx))", + (unsigned long long)blk, + (unsigned long long)owner, (unsigned long long)owner); + return 0; + } + savebh = bread(&sbd, blk); + if (savebh == NULL) + return 1; + err = save_bh(mfd, savebh, owner, blktype); brelse(savebh); - return blktype; + return err; } /* @@ -484,7 +492,7 @@ static void save_ea_block(struct metafd *mfd, struct gfs2_buffer_head *metabh, u b = (uint64_t *)(metabh->b_data); b += charoff + i; blk = be64_to_cpu(*b); - save_block(sbd.device_fd, mfd, blk, owner); + save_block(sbd.device_fd, mfd, blk, owner, NULL); } if (!ea.ea_rec_len) break; @@ -514,7 +522,7 @@ static void save_indirect_blocks(struct metafd *mfd, osi_list_t *cur_list, if (indir_block == old_block) continue; old_block = indir_block; - blktype = save_block(sbd.device_fd, mfd, indir_block, owner); + save_block(sbd.device_fd, mfd, indir_block, owner, &blktype); if (blktype == GFS2_METATYPE_EA) { nbh = bread(&sbd, indir_block); save_ea_block(mfd, nbh, owner); @@ -625,7 +633,7 @@ static void save_inode_data(struct metafd *mfd, uint64_t iblk) mybh = bread(&sbd, leaf_no); warm_fuzzy_stuff(iblk, FALSE); if (gfs2_check_meta(mybh, GFS2_METATYPE_LF) == 0) - save_block(sbd.device_fd, mfd, leaf_no, iblk); + save_block(sbd.device_fd, mfd, leaf_no, iblk, NULL); brelse(mybh); } } @@ -634,7 +642,7 @@ static void save_inode_data(struct metafd *mfd, uint64_t iblk) struct gfs2_buffer_head *lbh; lbh = bread(&sbd, inode->i_di.di_eattr); - save_block(sbd.device_fd, mfd, inode->i_di.di_eattr, iblk); + save_block(sbd.device_fd, mfd, inode->i_di.di_eattr, iblk, NULL); gfs2_meta_header_in(&mh, lbh); if (mh.mh_magic == GFS2_MAGIC && mh.mh_type == GFS2_METATYPE_EA) @@ -644,8 +652,8 @@ static void save_inode_data(struct metafd *mfd, uint64_t iblk) save_indirect_blocks(mfd, cur_list, lbh, iblk, 2, 2); else { if (mh.mh_magic == GFS2_MAGIC) /* if it's metadata */ - save_block(sbd.device_fd, mfd, - inode->i_di.di_eattr, iblk); + save_block(sbd.device_fd, mfd, inode->i_di.di_eattr, + iblk, NULL); fprintf(stderr, "\nWarning: corrupt extended " "attribute at block %llu (0x%llx) " @@ -722,7 +730,7 @@ static void save_allocated(struct rgrp_tree *rgd, struct metafd *mfd) for (j = 0; j < m; j++) { blk = ibuf[j]; warm_fuzzy_stuff(blk, FALSE); - blktype = save_block(sbd.device_fd, mfd, blk, blk); + save_block(sbd.device_fd, mfd, blk, blk, &blktype); if (blktype == GFS2_METATYPE_DI) save_inode_data(mfd, blk); } @@ -734,7 +742,7 @@ static void save_allocated(struct rgrp_tree *rgd, struct metafd *mfd) * If we don't, we may run into metadata allocation issues. */ m = lgfs2_bm_scan(rgd, i, ibuf, GFS2_BLKST_UNLINKED); for (j = 0; j < m; j++) { - save_block(sbd.device_fd, mfd, blk, blk); + save_block(sbd.device_fd, mfd, blk, blk, NULL); } } free(ibuf); @@ -826,7 +834,7 @@ void savemeta(char *out_fn, int saveoption, int gziplevel) exit(1); } /* Save off the superblock */ - save_block(sbd.device_fd, &mfd, GFS2_SB_ADDR * GFS2_BASIC_BLOCK / sbd.bsize, 0); + save_block(sbd.device_fd, &mfd, GFS2_SB_ADDR * GFS2_BASIC_BLOCK / sbd.bsize, 0, NULL); /* If this is gfs1, save off the rindex because it's not part of the file system as it is in gfs2. */ if (sbd.gfs1) { @@ -834,7 +842,7 @@ void savemeta(char *out_fn, int saveoption, int gziplevel) int j; blk = sbd1->sb_rindex_di.no_addr; - save_block(sbd.device_fd, &mfd, blk, blk); + save_block(sbd.device_fd, &mfd, blk, blk, NULL); save_inode_data(&mfd, blk); /* In GFS1, journals aren't part of the RG space */ for (j = 0; j < journals_found; j++) { @@ -842,7 +850,7 @@ void savemeta(char *out_fn, int saveoption, int gziplevel) for (blk = journal_blocks[j]; blk < journal_blocks[j] + gfs1_journal_size; blk++) - save_block(sbd.device_fd, &mfd, blk, blk); + save_block(sbd.device_fd, &mfd, blk, blk, NULL); } } /* Walk through the resource groups saving everything within */ @@ -861,7 +869,7 @@ void savemeta(char *out_fn, int saveoption, int gziplevel) for (blk = rgd->ri.ri_addr; blk < rgd->ri.ri_data0; blk++) { warm_fuzzy_stuff(blk, FALSE); - save_block(sbd.device_fd, &mfd, blk, blk); + save_block(sbd.device_fd, &mfd, blk, blk, NULL); } /* Save off the other metadata: inodes, etc. if mode is not 'savergs' */ if (saveoption != 2) { -- To stop receiving notification emails like this one, please contact the administrator of this repository.

1 0

[gfs2-utils] 13/38: gfs2_edit: Add rgrepair option
by git repository hosting 18 Apr '17

18 Apr '17

This is an automated email from the git hooks/post-receive script. andyp pushed a commit to branch RHEL7 in repository gfs2-utils. commit 16120985e3ab9b0ba2477c6c73206f82dfc3af85 Author: Bob Peterson <rpeterso(a)redhat.com> Date: Fri Dec 16 17:11:36 2016 -0600 gfs2_edit: Add rgrepair option This patch adds a new option, rgrepair, to gfs2_edit. Its purpose is to traverse all the resource groups in the rindex (assuming that the rindex is valid), and for each entry, verify it's really an rgrp, and that its bitmaps are bitmaps. If it finds an rgrp or bitmap block that's corrupt, it overwrites it with data that looks like a valid rgrp and/or bitmap. Then it looks for dinodes in that range of blocks. Any dinodes are marked as dinodes in the newly rewritten bitmap. Everything else is marked as a data block. That will allow fsck.gfs2 to try to straighten out the data from the free blocks. Signed-off-by: Bob Peterson <rpeterso(a)redhat.com> --- gfs2/edit/hexedit.c | 134 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 134 insertions(+) diff --git a/gfs2/edit/hexedit.c b/gfs2/edit/hexedit.c index 40a719e..cd3f084 100644 --- a/gfs2/edit/hexedit.c +++ b/gfs2/edit/hexedit.c @@ -2167,6 +2167,7 @@ static void usage(void) fprintf(stderr,"rgflags rgnum [new flags] - print or modify flags for rg #rgnum (0 - X)\n"); fprintf(stderr,"rgbitmaps <rgnum> - print out the bitmaps for rgrp " "rgnum.\n"); + fprintf(stderr,"rgrepair - find and repair damaged rgrp.\n"); fprintf(stderr,"-V prints version number.\n"); fprintf(stderr,"-c 1 selects alternate color scheme 1\n"); fprintf(stderr,"-d prints details (for printing journals)\n"); @@ -2259,6 +2260,135 @@ static void getgziplevel(char *argv[], int *i) (*i)++; } +static int count_dinode_blks(struct rgrp_tree *rgd, int bitmap, + struct gfs2_buffer_head *rbh) +{ + struct gfs2_buffer_head *tbh; + uint64_t b; + int dinodes = 0; + char *byte, cur_state, new_state; + int bit, off; + + if (bitmap) + off = sizeof(struct gfs2_meta_header); + else + off = sizeof(struct gfs2_rgrp); + + for (b = 0; b < rgd->bits[bitmap].bi_len << GFS2_BIT_SIZE; b++) { + tbh = bread(&sbd, rgd->ri.ri_data0 + + rgd->bits[bitmap].bi_start + b); + byte = rbh->b_data + off + (b / GFS2_NBBY); + bit = (b % GFS2_NBBY) * GFS2_BIT_SIZE; + if (gfs2_check_meta(tbh, GFS2_METATYPE_DI) == 0) { + dinodes++; + new_state = GFS2_BLKST_DINODE; + } else { + new_state = GFS2_BLKST_USED; + } + cur_state = (*byte >> bit) & GFS2_BIT_MASK; + *byte ^= cur_state << bit; + *byte |= new_state << bit; + brelse(tbh); + } + bmodified(rbh); + return dinodes; +} + +static int count_dinode_bits(struct gfs2_buffer_head *rbh) +{ + uint64_t blk; + struct gfs2_meta_header *mh = (struct gfs2_meta_header *)rbh->b_data; + char *byte; + int bit; + int dinodes = 0; + + if (be32_to_cpu(mh->mh_type) == GFS2_METATYPE_RG) + blk = sizeof(struct gfs2_rgrp); + else + blk = sizeof(struct gfs2_meta_header); + + for (; blk < sbd.bsize; blk++) { + byte = rbh->b_data + (blk / GFS2_NBBY); + bit = (blk % GFS2_NBBY) * GFS2_BIT_SIZE; + if (((*byte >> bit) & GFS2_BIT_MASK) == GFS2_BLKST_DINODE) + dinodes++; + } + return dinodes; +} + +static void rg_repair(void) +{ + struct gfs2_buffer_head *rbh; + struct rgrp_tree *rgd; + struct osi_node *n; + int b; + int rgs_fixed = 0; + int dinodes_found = 0, dinodes_total = 0; + + /* Walk through the resource groups saving everything within */ + for (n = osi_first(&sbd.rgtree); n; n = osi_next(n)) { + rgd = (struct rgrp_tree *)n; + if (gfs2_rgrp_read(&sbd, rgd) == 0) { /* was read in okay */ + gfs2_rgrp_relse(rgd); + continue; /* ignore it */ + } + /* If we get here, it's because we have an rgrp in the rindex + file that can't be read in. So attempt to repair it. + If we find a damaged rgrp or bitmap, fix the metadata. + Then scan all its blocks: if we find a dinode, set the + repaired bitmap to GFS2_BLKST_DINODE. Set all others to + GFS2_BLKST_USED so fsck can sort it out. If we set them + to FREE, fsck would just nuke it all. */ + printf("Resource group at block %llu (0x%llx) appears to be " + "damaged. Attempting to fix it (in reverse order).\n", + (unsigned long long)rgd->ri.ri_addr, + (unsigned long long)rgd->ri.ri_addr); + + for (b = rgd->ri.ri_length - 1; b >= 0; b--) { + int mtype = (b ? GFS2_METATYPE_RB : GFS2_METATYPE_RG); + struct gfs2_meta_header *mh; + + printf("Bitmap #%d:", b); + rbh = bread(&sbd, rgd->ri.ri_addr + b); + if (gfs2_check_meta(rbh, mtype)) { /* wrong type */ + printf("Damaged. Repairing..."); + /* Fix the meta header */ + memset(rbh->b_data, 0, sbd.bsize); + mh = (struct gfs2_meta_header *)rbh->b_data; + mh->mh_magic = cpu_to_be32(GFS2_MAGIC); + mh->mh_type = cpu_to_be32(mtype); + if (b) + mh->mh_format = + cpu_to_be32(GFS2_FORMAT_RB); + else + mh->mh_format = + cpu_to_be32(GFS2_FORMAT_RG); + bmodified(rbh); + /* Count the dinode blocks */ + dinodes_found = count_dinode_blks(rgd, b, rbh); + } else { /* bitmap info is okay: tally it. */ + printf("Undamaged. Analyzing..."); + dinodes_found = count_dinode_bits(rbh); + } + printf("Dinodes found: %d\n", dinodes_found); + dinodes_total += dinodes_found; + if (b == 0) { /* rgrp itself was damaged */ + rgd->rg.rg_dinodes = dinodes_total; + rgd->rg.rg_free = 0; + } + brelse(rbh); + } + rgs_fixed++; + } + if (rgs_fixed) + printf("%d resource groups fixed.\n" + "You should run fsck.gfs2 to reconcile the bitmaps.\n", + rgs_fixed); + else + printf("All resource groups are okay. No repairs needed.\n"); + exit(0); +} + /* ------------------------------------------------------------------------ */ /* parameterpass1 - pre-processing for command-line parameters */ /* ------------------------------------------------------------------------ */ @@ -2307,6 +2437,8 @@ static void parameterpass1(int argc, char *argv[], int i) termlines = 0; else if (!strcmp(argv[i], "rgflags")) termlines = 0; + else if (!strcmp(argv[i], "rgrepair")) + termlines = 0; else if (!strcmp(argv[i], "rg")) termlines = 0; else if (!strcasecmp(argv[i], "-x")) @@ -2483,6 +2615,8 @@ static void process_parameters(int argc, char *argv[], int pass) for (bmap = 0; bmap < rgd->ri.ri_length; bmap++) push_block(rgblk + bmap); } + else if (!strcmp(argv[i], "rgrepair")) + rg_repair(); else if (!strcasecmp(argv[i], "savemeta")) { getgziplevel(argv, &i); savemeta(argv[i+2], 0, gziplevel); -- To stop receiving notification emails like this one, please contact the administrator of this repository.

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

cluster-commits April 2017