LIO-Target on OpenSUSE 11.3

| 4 Comments
I mentioned I was playing with it, but now I have it working. So I'm sharing! Yay!

LIO-Target is one of several iSCSI modules available for Linux. As of the 2.6.38 kernel it'll be baked in. They even have a handy feature-comparison chart to explain why. For those with Microsoft or ESX environments, LIO-Target supports SCSI-3 persistent reservation, which is needed for clustering in both environments. It is nifty.

Disclaimer: There are some steps in this guide that I'm not going to give command-by-command guides to. If you don't know how to do that step you shouldn't be doing this at all. I know its unfriendly, but not being able to do that means you don't really know what you're doing and this kind of thing really isn't for you.

Anyway, this is how it works for 11.3, probably 11.2, and maybe not 11.4. 11.4 is still baking at the time of this post, and I'm reasonably certain that the LIO-Target stuff won't be mainline in time for feature freeze, but hey, won't know until it ships. It'll almost definitely be there for the next OpenSUSE version, be it 11.5 or 12.0.

Until such time as it becomes baked into OpenSUSE getting LIO-Target into it will require compiling custom kernel modules and hand editing certain key config files. Apparently there are some advanced UI tools available from Rising Tide systems, called 'rtsadmin', but I have not evaluated them.

In case you don't give a fig for this, I'm putting the guide under the fold.
  1. Install the kernel dev tools for OpenSUSE.
  2. Install other needed modules
  3. Download the LIO sources.
  4. Compile and install the LIO sources.
  5. Make sure the Target service starts.
  6. Modify your config files for sharing over iSCSI.
  7. Maintenance
Installing the Kernel development tools

This is as simple as installing the Kernel Development pattern in YaST or zypper. This will install the sources and enough of the software development environment to do Kernel compiles.

Install other needed modules

You also want to install the 'git' packages, the LIO sources are retrieved using that package, and the 'net-snmp-devel' package as that's used by the cli tools.

Download the LIO sources.

There are two packages you need to get them installed. Change to your directory of choice, and issue these commands:

git clone git://linux-iscsi.org/lio-core-backports.git lio-core-backports.git
git clone git://git.kernel.org/pub/scm/linux/storage/lio/lio-utils lio-utils.git
Compile and install the LIO sources.

The first thing you need to do is to go into your Kernel tree and prep it for module development. This is pretty simple.

make oldconfig
make prepare
If your kernel source directory is virgin, you don't have a .config file there already, the first will pull the /proc/config.gz file from your system and prepare it for compiling. If you DO already have  a .config file there, figure out whether or not you need to copy /proc/config.gz over it, or can re-use it (only you can make this decision), before doing "make oldconfig". The second will take the first step of compiling the kernel, which is needed for external module development.

Then change into the lio-core-backports.git directory, and issue the following commands:

./autoconfig
make
make install
If you haven't prepared the kernel, you'll get errors when you do the 'make'. Cross-compiling for another installed kernel is possible, but your easiest bet is to boot under that kernel before you any of this.

The last step will install the kernel modules, but you're not done quite yet. You need to install the lio-utils package.

CD into the lio-utils.git directory, and issue the same commands as the first package:

./autoconfig
make
make install
This will install the cli tools, /etc/init.d/target, and populate /etc/target with files. We'll get to those in a bit.

Make sure the Target service starts.

Go into YaST and turn the 'target' service on.

Modify your config-files for sharing over iSCSI.

The one down-side to these packages is that how the config files go together is not terribly intuitive. So I'll give you mine, and then explain what I'm doing and why. The command-reference, such as it is, can be found both here (pdf) and by issuing the usual help option to the commands in the script files.

/etc/target/tcm_start.sh
modprobe target_core_mod

## Set up HBAs
tcm_node --block iblock_0/Middling1 /dev/Data/MiddlingThing
tcm_node --block iblock_0/Weeny /dev/Data/WeenyThing1
These two commands set up a virtual HBA called iblock_0, and add two devices to that HBA. The final argument are the LVM volumes I'm associating with these virtual block-devices. They are unformatted. This also allows you to expand your eventual presented LUNs through normal LVM commands, and it should take live.

The other file, lio-start.sh is more complicated (long lines overflow, sorry about that).

# placeholder for lio_start.sh
### Start setting up the actual iSCSI share points

### Add virtual targets
# Target Group 1 (TPGT 1)
lio_node --addtpg iqn.2003-01.org.linux-iscsi.ac-cstore1.x86:sn.d66345278323 1
# Add network address to Target Portal Group
lio_node --addnp iqn.2003-01.org.linux-iscsi.ac-cstore1.x86:sn.d66345278323 1 172.16.232.129:3260
# Configure CHAP authentication for Target Portal Group
lio_node --disableauth iqn.2003-01.org.linux-iscsi.ac-cstore1.x86:sn.d66345278323 1

# Target Group 2 (TPGT 2)
lio_node --addtpg iqn.2003-01.org.linux-iscsi.ac-cstore1.x86:sn.d66345278323 2
lio_node --addnp iqn.2003-01.org.linux-iscsi.ac-cstore1.x86:sn.d66345278323 2 10.229.2.20:3260
lio_node --disableauth iqn.2003-01.org.linux-iscsi.ac-cstore1.x86:sn.d66345278323 2


### LUN: MiddlingThing, TargetGroup: 1
# Add the LUN
lio_node --addlun iqn.2003-01.org.linux-iscsi.ac-cstore1.x86:sn.d66345278323 1 0 MiddlingThingPort iblock_0/Middling1
# Create ACLs for the LUN
lio_node --addlunacl iqn.2003-01.org.linux-iscsi.ac-cstore1.x86:sn.d66345278323 1 iqn.1991-05.com.microsoft:tsmsbu2.univ.dir.wwu.edu 0 0

### LUN: FacShareLike, TargetGroup: 1
lio_node --addlun iqn.2003-01.org.linux-iscsi.ac-cstore1.x86:sn.d66345278323 1 1 FacShareLikePort iblock_0/FacshareLike
lio_node --addlunacl iqn.2003-01.org.linux-iscsi.ac-cstore1.x86:sn.d66345278323 1 iqn.1991-05.com.microsoft:tsmsbu2.univ.dir.wwu.edu 1 1

### LUN: Weeny1, TargetGroup: 2
lio_node --addlun iqn.2003-01.org.linux-iscsi.ac-cstore1.x86:sn.d66345278323 2 0 Weeny1Port iblock_0/Weeny
lio_node --addlunacl iqn.2003-01.org.linux-iscsi.ac-cstore1.x86:sn.d66345278323 2 iqn.1991-05.com.microsoft:msfsc-4.univ.dir.wwu.edu 0 0

### Enable all TargetGroups
lio_node --enabletpg iqn.2003-01.org.linux-iscsi.ac-cstore1.x86:sn.d66345278323 1
lio_node --enabletpg iqn.2003-01.org.linux-iscsi.ac-cstore1.x86:sn.d66345278323 2
The comments give a good idea as to what each step does, but here is the plain-english version of what I'm doing above.

Target Portal Group 1 is defined, and bound to the IP address and port combination of 172.16.232.129:3260, with adapter-level authentication turned off. I later add two LUNs to the Target Portal Group, and also give an ACL for each LUN that restricts what devices may connect to it.

Target Portal Group 2 is defined and bound to the IP address and port combination of 10.229.2.20:3260, also with adapter-level authentication turned off. I later add a single LUN to the Target Portal Group, which has its own ACL.

And finally I turn the Target Portal Groups on after they're fully configured.

The above scripts will execute each time the service starts, but dynamic reconfiguration can be done by running the commands individually. Unlike the IETD iSCSI daemon LIO-Target can be reconfigured on the fly. This is a very good thing since it looks like the Windows iSCSI initiator will not silently re-connect to an iSCSI target that suddenly disappears.

Maintenance.

Because you are compiling kernel modules outside of the OpenSUSE distribution channel, you will have to manually recompile the kernels every time a kernel-patch comes down the update pipe. That workflow will look similar to this:

  1. Apply the kernel patch, including the kernel-source patch.
  2. Go to your kernel source directory:
    1. make clean ; make oldconfig ; make prepare
  3. Go to your lio-core-backports.git directory
    1. ./autoconfig ; make ; make install
  4. Reboot
Since I haven't delt with a kernel update yet, step 4 may need to be moved between steps 1 and 2. No second reboot should be needed, just restart the 'target' service (/etc/init.d/target restart)

4 Comments

Feel free to delete this comment, but I just wanted to drop you a line to let you know that something appears amiss with your FeedBurner RSS feed... It doesn't seem to be available... If I remember correctly, it's been this way for a day or two.

8-)

I defined TPG1 in OpenFiler 2.99 following your example.
(No need to compile/install package to use lio-target in OF 2.99)

iSCSI at Windows Server 2003 can access the Target without issues.

Very good and very accurate script!!!!

I accessed this page in Opera and Chrome. Both have part of the commands "behind" the right hand frame.

Thanks a lot! A very good start of LIO for everyone.

While executing following command I am getting error.
Here /dev/dm-0 is LVM.


[root@ganesh-Studio-1558-relVM lio-utils.git]# tcm_node --block iblock_0/vol_1 /dev/dm-0
Traceback (most recent call last):
File "/usr/sbin/tcm_node", line 739, in
main()
File "/usr/sbin/tcm_node", line 715, in main
parser = OptionParser(version=tcm_version())
File "/usr/sbin/tcm_node", line 603, in tcm_version
return tcm_read("/sys/kernel/config/target/version").strip()
File "/usr/sbin/tcm_node", line 26, in tcm_read
with open(filename) as f:
IOError: [Errno 2] No such file or directory: '/sys/kernel/config/target/version'

If any body has the solution of this problem please let me know.