Make ext2fs work with partitions bigger than 2Gb.

This commit is port of large store patch from debian repository.
Additionally this patch was changed to use new libpager interface.

The main idea behind this patch is to use special disk cache up to 2Gb size
instead of mapping the whole partition. This cache is used to map only those
pages of partitions that are used at the moment or were used recently. This
allows to map only up to 2Gb of memory, although partition could be much
bigger.

This commit does not bring support for large files, only for partitions!

* ext2fs/balloc.c (ext2_free_blocks): Function changed to access disk blocks
in a new way.
(ext2_new_block): Likewise.
(ext2_count_free_blocks): Likewise.
(ext2_check_blocks_bitmap): Likewise.

* ext2fs/ext2fs.c (main): As now not the whole image is being mapped, but
there is disk cache is being used, appropriately were changed some names of
functions and variables.
(diskfs_reload_global_state): Likewise.

* ext2fs/ext2fs.h (DISK_CACHE_BLOCKS): New macro. Sets size of disk cache.
(DC_INCORE, DC_UNTOUCHED, DC_FIXED, DC_DONT_REUSE, DC_NO_BLOCK): New macros,
used for keeping status of block in disk cache.
(DISK_CACHE_LAST_READ_XOR) [NDEBUG]: New macro for debugging purposes.
(struct disk_cache_info): Structure that keeps information about block in
disk cache.
(bptr_index): New macro for converting address in memory to index of block
in disk cache.

(boffs_ptr): Macro has been converted to function to take into account
presence of disk cache.
(bptr_offs): Likewise.

(dino -> dino_ref): Function has been renamed and changed to take into
account presence of disk cache.
(dino_deref): New function that releases dinode.

(record_global_poke): Function changed to access disk blocks in a new way.
(sync_global_ptr): Likewise.
(record_indir_poke): Likewise.

(sync_global): Add debugging macro.

* ext2fs/getblk.c (ext2_alloc_block): Function changed to access disk block
in a new way.
(block_getblk): Likewise.

* ext2fs/hyper.c (get_hypermetadata): Function has been changed because,
to read sblock on its own.
(map_hypermetadata): New function that sets sblock and group_desc_image as
pointers in disk cache.
(diskfs_set_hypermetadata): Increase reference count in disk cache for
sblock, when it is poked.
(diskfs_readonly_changed): Update function to use disk cache.

* ext2fs/ialloc.c (diskfs_free_node): Function changed to access disk blocks
in a new way.
(ext2_alloc_inode): Likewise.
(ext2_count_free_inodes): Likewise.
(ext2_check_inodes_bitmap): Likewise.

* ext2fs/inode.c (diskfs_cached_lookup): Use disk cache instead of disk
image.

(read_node): Function changed to access disk blocks using interface. Also
debugging macro has been added.
(write_node): Likewise.

(diskfs_set_translator): Function changed to access disk blocks using
interface.
(diskfs_set_translator): Likewise.

* ext2fs/pager.c (disk_image): Obsolete variable removed.
(STAT_ADD): New macro.
(FREE_PAGE_BUFS): Remove obsolete macro.
(get_page_buf -> get_buf): Function renamed and changed to return buffer of
asked size.
(free_page_buf -> free_buf): Function renamed and changed.

(file_pager_read_page -> file_pager_read): Function renamed and changed to
process several pages at once.
(file_pager_write_page -> file_pager_write): Likewise.
(disk_pager_read_page -> disk_pager_read): Likewise.
(disk_pager_write_page -> disk_pager_write): Likewise.

(pending_blocks_write): Function has been changed.
(ext2_pager_notify_evict): New function.

(pager_read_page -> ext2_read_pages): Function has been renamed and changed
to support new libpager interface.
(pager_write_page -> ext2_write_pages): Likewise.
(pager_unlock_page -> ext2_unlock_pages): Likewise.
(pager_report_extent -> ext2_report_extent): Likewise.
(pager_clear_user_data -> ext2_clear_user_data): Likewise.

(pager_dropweak): Remove obsolete function.

(ext2_ops): New variable.
(disk_cache): Likewise.
(disk_cache_size): Likewise.
(disk_cache_blocks): Likewise.
(disk_cache_bptr): Likewise.
(disk_cache_info): Likewise.
(disk_cache_hint): Likewise.
(disk_cache_lock): Likewise.
(disk_cache_reassociation): Likewise.

(disk_cache_init): New function.
(disk_cache_return_unused): Likewise.
(disk_cache_block_ref_no_block): Likewise.
(disk_cache_block_is_cached): Likewise.
(disk_cache_block_ref): Likewise.
(disk_cache_block_ref_hint_no_block): Likewise.
(disk_cache_block_ref_ptr): Likewise.
(disk_cache_block_deref): Likewise.
(disk_cache_block_is_ref): Likewise.

(create_disk_pager): Function has been changed to support both new
interfaces of libpager and disk cache.
(diskfs_get_filemap): Function has been changed to support new interface of
libpager.

* ext2fs/pokel.c (pokel_add): Function changed to take into account new way
of working with disk blocks.
(__pokel_exec): Likewise.

* ext2fs/truncate.c (trunc_indirect): Function changed to take into account
both new way of working with disk blocks and changes in libpager.
(force_delayed_copies): Function changed to support new libpager interface.

1 files changed, 30 insertions, 19 deletions

diff --git a/ext2fs/inode.c b/ext2fs/inode.c
index bda964fc..56311e29 100644
--- a/ext2fs/inode.c
+++ b/ext2fs/inode.c
@@ -92,7 +92,7 @@ diskfs_cached_lookup (ino_t inum, struct node **npp)
   dn->dir_idx = 0;
   dn->pager = 0;
   rwlock_init (&dn->alloc_lock);
-  pokel_init (&dn->indir_pokel, diskfs_disk_pager, disk_image);
+  pokel_init (&dn->indir_pokel, diskfs_disk_pager, disk_cache);
 
   /* Create the new node.  */
   np = diskfs_make_node (dn);
@@ -201,13 +201,17 @@ read_node (struct node *np)
   error_t err;
   struct stat *st = &np->dn_stat;
   struct disknode *dn = np->dn;
-  struct ext2_inode *di = dino (np->cache_id);
+  struct ext2_inode *di;
   struct ext2_inode_info *info = &dn->info;
 
+  ext2_debug ("(%d)", np->cache_id);
+
   err = diskfs_catch_exception ();
   if (err)
     return err;
 
+  di = dino_ref (np->cache_id);
+
   st->st_fstype = FSTYPE_EXT2FS;
   st->st_fsid = getpid ();	/* This call is very cheap.  */
   st->st_ino = np->cache_id;
@@ -285,6 +289,7 @@ read_node (struct node *np)
       info->i_high_size = di->i_size_high;
       if (info->i_high_size)	/* XXX */
 	{
+	  dino_deref (di);
 	  ext2_warning ("cannot handle large file inode %Ld", np->cache_id);
 	  return EFBIG;
 	}
@@ -307,20 +312,12 @@ read_node (struct node *np)
     }
   dn->info_i_translator = di->i_translator;
 
+  dino_deref (di);
   diskfs_end_catch_exception ();
 
   if (S_ISREG (st->st_mode) || S_ISDIR (st->st_mode)
       || (S_ISLNK (st->st_mode) && st->st_blocks))
-    {
-      unsigned offset;
-
-      np->allocsize = np->dn_stat.st_size;
-
-      /* Round up to a block multiple.  */
-      offset = np->allocsize & ((1 << log2_block_size) - 1);
-      if (offset > 0)
-	np->allocsize += block_size - offset;
-    }
+    np->allocsize = round_block (np->dn_stat.st_size);
   else
     /* Allocsize should be zero for anything except directories, files, and
        long symlinks.  These are the only things allowed to have any blocks
@@ -408,7 +405,9 @@ write_node (struct node *np)
 {
   error_t err;
   struct stat *st = &np->dn_stat;
-  struct ext2_inode *di = dino (np->cache_id);
+  struct ext2_inode *di;
+
+  ext2_debug ("(%d)", np->cache_id);
 
   if (np->dn->info.i_prealloc_count)
     ext2_discard_prealloc (np);
@@ -425,6 +424,8 @@ write_node (struct node *np)
       if (err)
 	return NULL;
 
+      di = dino_ref (np->cache_id);
+
       di->i_generation = st->st_gen;
 
       /* We happen to know that the stat mode bits are the same
@@ -505,6 +506,7 @@ write_node (struct node *np)
       diskfs_end_catch_exception ();
       np->dn_stat_dirty = 0;
 
+      /* Leave invoking dino_deref (di) to the caller.  */
       return di;
     }
   else
@@ -664,7 +666,7 @@ diskfs_set_translator (struct node *np, const char *name, unsigned namelen,
   if (err)
     return err;
 
-  di = dino (np->cache_id);
+  di = dino_ref (np->cache_id);
   blkno = di->i_translator;
 
   if (namelen && !blkno)
@@ -677,6 +679,7 @@ diskfs_set_translator (struct node *np, const char *name, unsigned namelen,
 			0, 0, 0);
       if (blkno == 0)
 	{
+	  dino_deref (di);
 	  diskfs_end_catch_exception ();
 	  return ENOSPC;
 	}
@@ -700,15 +703,20 @@ diskfs_set_translator (struct node *np, const char *name, unsigned namelen,
       np->dn_stat.st_mode &= ~S_IPTRANS;
       np->dn_set_ctime = 1;
     }
+  else
+    dino_deref (di);
 
   if (namelen)
     {
+      void *blkptr;
+
       buf[0] = namelen & 0xFF;
       buf[1] = (namelen >> 8) & 0xFF;
       memcpy (buf + 2, name, namelen);
 
-      bcopy (buf, bptr (blkno), block_size);
-      record_global_poke (bptr (blkno));
+      blkptr = disk_cache_block_ref (blkno);
+      memcpy (blkptr, buf, block_size);
+      record_global_poke (blkptr);
 
       np->dn_stat.st_mode |= S_IPTRANS;
       np->dn_set_ctime = 1;
@@ -726,7 +734,7 @@ diskfs_get_translator (struct node *np, char **namep, unsigned *namelen)
   error_t err = 0;
   daddr_t blkno;
   unsigned datalen;
-  const void *transloc;
+  void *transloc;
 
   assert (sblock->s_creator_os == EXT2_OS_HURD);
 
@@ -734,9 +742,11 @@ diskfs_get_translator (struct node *np, char **namep, unsigned *namelen)
   if (err)
     return err;
 
-  blkno = (dino (np->cache_id))->i_translator;
+  struct ext2_inode *di = dino_ref (np->cache_id);
+  blkno = di->i_translator;
+  dino_deref (di);
   assert (blkno);
-  transloc = bptr (blkno);
+  transloc = disk_cache_block_ref (blkno);
 
   datalen =
     ((unsigned char *)transloc)[0] + (((unsigned char *)transloc)[1] << 8);
@@ -751,6 +761,7 @@ diskfs_get_translator (struct node *np, char **namep, unsigned *namelen)
 	memcpy (*namep, transloc + 2, datalen);
     }
 
+  disk_cache_block_deref (transloc);
   diskfs_end_catch_exception ();
 
   *namelen = datalen;