##### ##### Setting up tbdb for a new boss node ##### Note: we are working on better automating many of the procedures in this document - for now, though, many of them are manual. Note: steps labeled "Local Only" are only required when setting up a testbed with local nodes - they can be skipped in a widearea-only testbed. ##### Step 1 - Setup users At this point, it's a good idea to get the nfs mounts between boss and ops setup, as well at the ssh root keys. See the boss and ops setup documents for details. Run the 'firstuser' script. This will get put in the utils subdirectory of your object tree. (Note: configure will not set the executable bit on the script, so you'll need to set it yourself, or run 'perl firstuser'.) This script will let you create the first user and project in the database. By default, it will grab information about the account it's being run as (probably you,) or you can supply a username, an encrypted password string (grab it from a password file,) etc. This script will also prompt you to create the first project - create the project 'emulab-ops', which is a meta-project for holding experiments. If you want to make sure that this account is a member of any Unix groups (such as wheel,) run 'unixgroups -a ...' ##### Step 2 - Setup projects and experiments At this point, you should get the webserver up and running on your boss node. Again, instructions are in the boss setup documentation. Go to the web interface, and log in as the admin account you created. Create a project for the administrators at your site (we call ours 'testbed', you can call yours whatever you call your project or group). As soon as you've sent the request to sart the project, you can use the 'New Project Approval' link to make it active. Others at your site can now apply to join your project. There are three experiments that must be created in the emulab-ops project. Just start them through the normal 'begin experiment' page, and don't give an NS file. These experiments are: hwdown - Nodes that are down due to hardware failure reloading - Nodes that are having their disks automatically reloaded reloadpending - Nodes that are awaiting disk reloading Any other 'holding' experiments that you want for operations should be put into the emulab-ops project. ##### Step 3 - Setup web sql editor Several of the steps below require you to add data to the database by hand. If you're comfortable with SQL, you can do this directly with SQL queries, or you can use the generic web-based SQL table editor provided with the testbed software. If you plan to use the former method, you can skip this step. ********************************** WARNING ********************************** * Many tables depend on data in other tables, or depend on programs running * * to effect a change. Thus, you should not edit tables other than the ones * * described in this document. * * You have been warned...... * ********************************** WARNING ********************************** First, you'll want to protect the webdb script from outside browsers. Because of its flexibility, it would be quite dangerous if it were broken into. So, we add an additional layer of protection by limiting the IP addresses it may be used from. Open your httpd.conf file (created as part of the boss installation), and find the 'Directory' directives. Add a section such as this: AllowOverride None order deny,allow deny from all allow from 155.99.212. If you installed the testbed tree somewhere other than /usr/testbed, fix the directory. Change the 'allow from' line to match your IP subnet (note the '.' on the end of the address, to match the entire subnet). You can have as many 'allow' lines as you want. Restart apache. Next, you'll need to specify which users have the ability to edit the database. This is done with the 'dbedit' column in the users table. When we automate user creation, we'll add a way to set this when the user is created. Now, you'll need to specify which tables are editable from the web interface. This is to protect against accidental database damage, as mentioned in the warning above. We'll soon provide contents for this table (webdb_table_permissions) with the testbed software. ##### Step 4 - Setup switches ##### Local Only 1) Create node types for switches Add entries to the node_types table for each type of switch you'll be using. The class column should be 'switch'. The type for Cisco switches should be something like 'cisco6509', or 'catalyst2950'. Most of the other columns aren't important for switches (so you can leave them blank), but putting in max_cards (if the switch is expandable) and max_ports is a good idea. 2) Create interface types for switch interconnects (if any) If you'll be connecting switches together, also add interface types for the ports you'll be using to link them together. Unlike the logical IP interfaces, make sure that you get physical information like speed (in Kbps) and duplex are correct. 3) Create switches in nodes table Next, add the switches to the nodes table. The only necessary fields here are node_id (choose one appropriate to the switch), type (use one of the ones you created earlier,) and role. Role, for switches, should be either 'testswitch' or 'controlswitch', depending on whether you're using it for the experimental or control network. 4) Add state for switch interconnects If you'll be connecting switches together, add interfaces to each of them. Unlike the previous interfaces, make sure that you get the correct card and port numbers. For switches that do not have expansion modules, the card number is assumed to be 1. Make sure to get the current_speed (this time, in Mbps) and duplex correct. Now, go into the wires table and add wires connecting the switch interfaces you just created. Make sure to set the type to 'Trunk'. In this case, it doesn't matter which switch you use as node 1 (the node_id1, card1, port1 columns), and which you use as node 2. If you have a non-modular switch, all ports are considered to be on card 0. 7) Create switch stacks Into the switch_stack_types table, create entries for each stack in your network. A stack is set of switches that share common VLANs. Usually, your experimental switches will be one stack, and the control network another. You should create a stack even if there is only one switch in it. The stack should be named (the stack_id column) after the master switch in the stack. In our case, for example, our experimental network stack is called 'cisco1'. The stack_type column is used by snmpit to determine which module to use to talk to the stack. The currently supported values are 'cisco' and 'intel'. Making a stack for the control network is optional. There are a few columns in this table that you may want to set: supports_private: (Cisco only) This switch can make private VLANs - probably best to leave it as 0 single_domain: (Cisco only) Means that all switches in the stack use VTP to maintain a common set of VLANs. This means that we will only create VLANs on the stack leader - if set to 0, we create VLANs on each switch individually snmp_community: The SNMP community string we use for read-write access to the switch. Most switches use 'private' by default. min_vlan: The smallest VLAN number that the testbed software will use max_vlan: Ditto, but the biggest number Finally, add switches to these stacks by putting entries in the switch_stacks table, listing the node_id of each switch and the stack_id of the stack it belongs to. ##### Step 5 - Setup control hardware ##### Local Only 1) Boss/ops nodes Optional step: You can put your control (boss and ops) nodes in the database if you wish. You'll need to add entries to the nodes, interfaces, and wires tables, and possibly the node_types and interface_types tables as well. See the instructions for adding nodes below for descriptions of the contents of these tables. 2) Power controllers The only two supported power controllers at this time are the APC AP9210 and the BayTech RPC27. Theoretically, other SNMP and serial-controlled power controllers should not be hard to support. If your power controllers are controlled via IP and SNMP, add a node type ander interface type for them. Then, give them an entries in the nodes, interfaces, and wires tables. Make sure to get the IP address correct in the interfaces table. The role listed in the nodes table should be 'powerctrl'. If you have serial power controllers, just create entries node_types and nodes - no interfaces or wires. Again, the role in the nodes table should be 'powerctrl'. ##### Step 6 - Images and OSids These will depend on the image(s) and any OSKit kernels you've recieved, or built yourself. Since you're probably using disk images from Utah, the best thing to do is ask us for the appropriate lines for the os_info and images tables. Make sure to get the PXEBOOT, PXEFRISBEE, and PXEFBSD OSIDs, which are required to load disk images. For a widearea-only testbed, no images entries in the images table a required, but OSIDs are still useful, to determine what a node's capabilities are. ##### Step 7 - Setup nodes To add nodes to the testbed, see setup-nodes.txt .