Sunday 29 July 2012

Writing Your Own Device Mapper Target

Device Mapper


In the Linux kernel, the device-mapper serves as a generic framework to map a virtual layer of block devices onto existing block devices. The device mapper framework promotes a clean separation of policy and mechanism between user and kernel space respectively. It forms the foundation of LVM2 and EVMS, software RAIDs, dm-crypt disk encryption, and offers additional features such as file-system snapshots. Device-mapper works by processing data passed in from virtual block devices, that it itself provides, and then passing the resultant data on to existing block devices.

"Device mapper can be defined as a generic way to add required functionality in the storage stack by creating virtual layer of block devices and mapping them to existing block devices ". It create virtual layers of block devices that can do different things on top of existing underlying block devices like striping, concatenation, mirroring, snapshot, etc.

The Device Mapper is a modular kernel driver that provides a generic framework for volume management. It has been introduced in the kernel version 2.6. The device-mapper is used by the LVM2 and EVMS 2.x tools.

Device Mapper Target


As stated above, we can create various logical layers through device mapper to carry out the required functionality. Each such layer is created by defining “a device mapper target” for that layer.

There is one to one correspondence between a virtual layer at device mapper layer and the dm target for that layer. The particular dm target contains the code which does the task of implementing functionality which the virtual layer intends to do. For example, a device mapper target can be written to implement mirroring over existing block devices. This dm target shows a virtual layer to upper layers which do the task of mirroring.

Currently seven such features have been added to device mapper through device mapper targets. The device mapper targets are as follows:
  • Linear 
  • RAID-0 / Striped 
  • RAID-1 / Mirrored RAID 
  • Snapshot 
  • DM-Crypt

Write Our Own Device Mapper Target

Our device mapper target is going to be a kernel module. Lets say we call our dm target as 'basic_target'
and the corresponding file is basic_target.c.

basic_target.c

#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/bio.h> #include <linux/device-mapper.h>



/* This is a structure which will store  information about the underlying device 
*  Param:
* dev : underlying device
* start:  Starting sector number of the device
*/


struct my_dm_target {
        struct dm_dev *dev;
        sector_t start;
};



/* This is map function of basic target. This function gets called whenever you get a new bio
 * request.The working of map function is to map a particular bio request to the underlying device. 
 *The request that we receive is submitted to out device so  bio->bi_bdev points to our device.
 * We should point to the bio-> bi_dev field to bdev of underlying device. Here in this function,
 * we can have other processing like changing sector number of bio request, splitting bio etc. 
 *
 *  Param : 
 *  ti : It is the dm_target structure representing our basic target
 *  bio : The block I/O request from upper layer
 *  map_context : Its mapping context of target.
 *
 *: Return values from target map function:
 *  DM_MAPIO_SUBMITTED :  Your target has submitted the bio request to underlying request
 *  DM_MAPIO_REMAPPED  :  Bio request is remapped, Device mapper should submit bio.  
 *  DM_MAPIO_REQUEUE   :  Some problem has happened with the mapping of bio, So 
 *                                                re queue the bio request. So the bio will be submitted 
 *                                                to the map function  
 */

static int basic_target_map(struct dm_target *ti, struct bio *bio,union map_info *map_context)
{
        struct my_dm_target *mdt = (struct my_dm_target *) ti->private;
        printk(KERN_CRIT "\n<<in function basic_target_map \n");

        bio->bi_bdev = mdt->dev->bdev;

        if((bio->bi_rw & WRITE) == WRITE)
                printk(KERN_CRIT "\n basic_target_map : bio is a write request.... \n");
        else
                printk(KERN_CRIT "\n basic_target_map : bio is a read request.... \n");
        submit_bio(bio->bi_rw,bio);

        printk(KERN_CRIT "\n>>out function basic_target_map \n");      
        return DM_MAPIO_SUBMITTED;
}

/* This is Constructor Function of basic target * Constructor gets called when we create some device of type 'basic_target'. * So it will get called when we execute command 'dmsetup create' * This function gets called for each device over which you want to create basic * target. Here it is just a basic target so it will take only one device so it * will get called once. */

static int basic_target_ctr(struct dm_target *ti,unsigned int argc,char **argv)
{
        struct my_dm_target *mdt;
        unsigned long long start;

        printk(KERN_CRIT "\n >>in function basic_target_ctr \n");

        if (argc != 2) {
                printk(KERN_CRIT "\n Invalid no.of arguments.\n");
                ti->error = "Invalid argument count";
                return -EINVAL;
        }

        mdt = kmalloc(sizeof(struct my_dm_target), GFP_KERNEL);

        if(mdt==NULL)
        {
                printk(KERN_CRIT "\n Mdt is null\n");
                ti->error = "dm-basic_target: Cannot allocate linear context";
                return -ENOMEM;
        }      

        if(sscanf(argv[1], "%llu", &start)!=1)
        {
                ti->error = "dm-basic_target: Invalid device sector";
                goto bad;
        }

        mdt->start=(sector_t)start;

/* dm_get_table_mode * Gives out you the Permissions of device mapper table. * This table is nothing but the table which gets created * when we execute dmsetup create. This is one of the * Data structure used by device mapper for keeping track of its devices. * * dm_get_device * The function sets the mdt->dev field to underlying device dev structure. */
   
        if (dm_get_device(ti, argv[0], dm_table_get_mode(ti->table), &mdt->dev)) {
                ti->error = "dm-basic_target: Device lookup failed";
                goto bad;
        }

        ti->private = mdt;

        printk(KERN_CRIT "\n>>out function basic_target_ctr \n");                      
        return 0;

  bad:
        kfree(mdt);
        printk(KERN_CRIT "\n>>out function basic_target_ctr with errorrrrrrrrrr \n");          
        return -EINVAL;
}

/* * This is destruction function * This gets called when we remove a device of type basic target. The function gets * called per device. */
static void basic_target_dtr(struct dm_target *ti)
{
        struct my_dm_target *mdt = (struct my_dm_target *) ti->private;
        printk(KERN_CRIT "\n<<in function basic_target_dtr \n");        
        dm_put_device(ti, mdt->dev);
        kfree(mdt);
        printk(KERN_CRIT "\n>>out function basic_target_dtr \n");              
}

/*
* This structure is fops for basic target.
*/
static struct target_type basic_target = {
       
        .name = "basic_target",
        .version = {1,0,0},
        .module = THIS_MODULE,
        .ctr = basic_target_ctr,
        .dtr = basic_target_dtr,
        .map = basic_target_map,
};
       
/*---------Module Functions -----------------*/

static int init_basic_target(void)
{
        int result;
      result = dm_register_target(&basic_target);
        if(result < 0)
                printk(KERN_CRIT "\n Error in registering target \n");
        return 0;
}

static void cleanup_basic_target(void)
{
        dm_unregister_target(&basic_target);
}
module_init(init_basic_target);
module_exit(cleanup_basic_target);
MODULE_LICENSE("GPL");

Makefile 


obj-m +=basic_target.o

all:
        make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules

clean:
        make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean

USAGE

TBD 
http://linuxgazette.net/114/kapil.html



Stay tuned to http://techgmm.blogspot.in/
Thanks !!!

11 comments:

Mahesh said...

Hi,
I just have a couple of doubts :

1. Are the ctr and dtr functions called when the target is registered/unregistered respectively ?

2. What do dm_get_device and dm_get_table_mode do ?

Gaurav Mahajan said...

Hi Mahesh,
Answers
1. The new device mapper target gets registered with Device Mapper when we insert the module. So in function 'init_basic_target' the function 'dm_register_target' gets called which intern registers our basic_target with the device mapper. just to check if your target is registered you can check it with 'dmsetup targets' command. This command displays the targets registered with device mapper.

The ctr i.e Constructor gets called when we create some device of type 'basic_target'. So it will get called when we execute command 'dmsetup create ' It will be added soon in usage part.

2. dm_get_table_mode
Gives out you the Permissions of device mapper table. This table is nothing but the table which gets created when we execute dmsetup create. This is one of the Data structure used by device mapper for keeping track of its devices.

dm_get_device
Device mapper creates a layer over underlying devices. This function gives the data structure 'dm_dev' which keeps track of the underlying devices. So whenever a request comes to basic_target that request's bdev field points to basic_targets device. We have to map that request to underlying device. So there we need the information of underlying device. This information is stored in mdt structure.
So this function create appropriate data structure to keep track of underlying devices.

Chetan Balshetwar said...

Nice blog man keep going.........

Z said...

Hi, I just have a few very basic doubts:-


1. What to do after doing insmod basic_target.ko?
2. How to use the dmsetup create command?
3. How do I start using the dm layer?

Thanks in advance. :)

Gaurav Mahajan said...

Hi Zishan,

1. What to do after doing insmod basic_target.ko (your code from your blog) ?

dmsetup targets

This will show that list of all registered targets with device mapper.
so here you should see the basic_ target in the list.
this confirms that basic target is registered with DM.

2. How to use the dmsetup create command?

Suppose you want to create a basic target on /dev/loop6 and loop6 is of size 2gb

echo 0 20000 basic_target /dev/loop6 0 | dmsetup create new_device

Here first 0 is start of device, 20000 is size of device, basic_target is a type of dm device, last 0 is start of /dev/loop6.

new_device gets create in /dev/mapper directory.
You can check with dmsetup ls --tree

3. How do i start using the dm layer?

now you have /dev/mapper/new_device.
You can create a file system on it and start using it.

When you do any io on thi device you will see the function call trace in /var/log/messages.

Unknown said...

Hi Mahesh...
Nice work dude.
I have some doubts:
1. How can I select a particular device from the map table?
2. What are arguments passed in the argv?
3. Can control the arguments passed?
Thanks in advance

Gaurav Mahajan said...

Hi,
1. How can I select a particular device from the map table?
I am not so sure how we can select particular device from the target. But we can maintain the list of devices when our constructor gets called.

2. What are arguments passed in the argv?

These are nothing but the starting sector of your device and size of the device. You can have more arguments as per you own requirement.


3. Can control the arguments passed?

yes you can add the arguments and they will be available in argv in the constructor.

Thanks

Unknown said...

Nice explanation.

777777777777777 said...

sudo echo 0 21474836480 basic_target /dev/sdc 0|dmsetup create my_basic_target_device

/dev/mapper/control: open failed: Permission denied
Failure to communicate with kernel device-mapper driver.

Incompatible libdevmapper (unknown version) and kernel driver (unknown version).
Command failed

Gaurav Mahajan said...

Probably you are running with kernel without device mapper in it. Try enabling it while configuring your custom kernel or install dm modules and see if they are loaded.

for initial troubleshooting make sure this works
$ sudo dmsetup ls
No devices found
$

Unknown said...

Is it possible to change data storaged in bio? For example, to encode it with XOR.