-
Notifications
You must be signed in to change notification settings - Fork 8
The CTP6
[[TOC]]
uhal Quick Tutorial: https://svnweb.cern.ch/trac/cactus/wiki/uhalQuickTutorial#uHALQuickTutorial
There are three things to setup to read out the CTP6 front-end:
- the
ctp6_fe_uart_ipbuspayload running on the front end FPGA via XMD - the
softipbus-forwardbinary must be running on the backend FPGA Linux - the
ctp6commander/cli.pyclient is then used to query memory on the backend FPGA.
Technically, the cli.py sends a uHAL TCP request to a server (softipbus-forward) that is running on the backend linux. This request is then forwarded over the UART (serial device) that is connected between the back end and the front-end FPGAs. The front-end then reads the IPBus packet stream from the serial device, parses the packets, and then sends the response packets back along the serial line.
On a machine connected via JTAG to the CTP6:
cd ~/trigger_code/cms-calo-layer1
source environment.sh
make
cd ctp6_fe_uart_ipbus
make upload
the payload should now be running on the front-end. (Until a power cycle, or crash). To verify that you see the correct output on screen check here: https://github.com/uwcms/ctp6commander/blob/master/docs/Load_FE_Payload
On a machine connected via ethernet to the uTCA crate holding the CTP6:
cd ~/trigger_code/cms-calo-layer1
source environment.sh
cd ../softipbus
# Print out a lot of info to the console. Can set to 0 to silence.
export LOG_LEVEL=3
make
# Set the IP address for the target CTP (change 33 to appropriate value)
export CTPIP=192.168.1.33
# Copy the binaries into the CTP via scp
make upload2ctp
make forwardonctp
# this will:
# a. open an SSH session forwarding port 60001 to the local computer
# b. kill any existing softipbus-forward servers
# c. start a new server from the binary uploaded in the previous step
#After you are finished make sure to close the port and clear the /tmp/softipbus-forward file
make cleanonctp
To verify that you have the 'correct' screen output see here: https://github.com/uwcms/ctp6commander/blob/master/docs/Make_And_Upload_toCTP
Possible error message: "ERROR: Unable to bind address: 98 (Address already in use)". To verify the port is (or is not) in use, use ssh to log into the CTP6 via root@192.168.1.33 and run netstat -tulpn look for the port (60001 or 60003). You can kill it using kill+PID number OR
manually kill previous forwarding from a machine which can connect to the CTP6 via ethernet:
make cleanonctp
#Wait ~1-2 minutes
make forwardonctp
On the machine that is connected via ethernet to the CTP6, we can query the link status of links 1, 5, and 12 by running the following:
cd ctp6commander
source environment.sh
/usr/bin/python cli.py --connection ctp6_connections.xml --verbose status 1 5 12
To verify that you have a 'correct' screen output check here: https://github.com/uwcms/ctp6commander/blob/master/docs/Using_CLI
- XMD% ERROR: Failed to Open JTAG Cable Cable target is not connected to the host
Rebooting the pc will change the permissions of the drivers, making JTAG inaccessible.
sudo chmod 666 /dev/windrvr6
(See /opt/Xilinx/14.4/ISE_DS/common/bin/lin/install_script/install_drivers/readme.txt)
Log into the linux back end, and run: cat /var/log/clock_init.log, which should output:
SETTING REGISTER MAP
REGISTER MAP IS SET
Waiting for Input Clock...
Detected.
Waiting for PLL Lock...
Acquired.
Configuration Complete.
If it doesnt, run /opt/AMC13/ecad_activate_amc13.sh 16 from the PC connected with the loopback cable to the crate (currently l1ts-rct-01), and check again. If we get a new AMC13, it will have a different serial number, the current serial number is on a sticker on the front of the AMC13.
The new image on the boards should automatically configure the amplitude to 7 and the deemphasis to 0 at startup. You can change these from the linux after boot with
$ avago_config 7 0
This is needed to update the backend AND frontend bitfiles. It is persistent.
First, login to l1ts-rct-02, since it has the USB hub hooked up to it.
Hook up the USB hub to the port on the front of the CTP6 (it may be plugged
into the oRSC). Type dmesg and look at the end to see the name of the
/dev/ttyUSBXX that has just appeared.
On l1ts-rct-02, run
sudo screen /dev/ttyUSB0 9600
Now pull out the handle on the card, light should turn blue, then push it back in. Light should be back off. The CTP will start booting on your serial console, interrupt the boot process.
At u-boot prompt:
Enter command bootm e1800000 to boot maintenance mode.
Now login as root on the serial console (l1ts-rct-01), and run ifconfig for the IP address.
Now the copy the top_prom.bin file to the card:
scp cms-calo-layer1/bitfiles/ctp6/top_prom.bin root@192.168.1.33:/tmp
Log in to the card via console or "ssh root@192.168.1.33" (also from l1ts-rct-01), and execute the following:
flash_eraseall /dev/flash/fpga
dd if=/tmp/top_prom.bin of=/dev/flash/fpga
Now go restart the card by pulling the lever and pushing it back in.
This is important if you want to have sane behavior while accessing /dev/ttyUL1 using shell tools on the backend cat, echo, etc. Needed for ctp6_fe_uart_echo_test to behave correctly.
stty -F /dev/ttyUL1 raw -echoNote: This may also work with a second Network Interface Card (NIC).
- Log into cms904usr then to the PC with the crossover cable in the second ethernet connection (eth1), currently l1ts-rct-01.
- For short term operation:
sudo /sbin/ifconfig eth1 192.168.1.2 - For more permanent operation sudo system-config-network. Then "Edit a device params". Choose eth1 and set it up as follows:
Eliminate DHCP IP 192.168.1.2 Netmask 192.186.1.255 - To test connection to the MCH ping 192.168.1.240
- To log in to the CTP6: ssh root@192SPAMNOT.168.1.33 ask Pam or Maria for the password.
First, make sure that the gray USB cable running from the l1rct-ts-01 is plugged into the red Xilinux JTAG programmer which is hooked up to the CTP6. If it is plugged in and the uTCA crate is powered on, the status light on the JTAG programmer should be green.
Login to l1ts-rct-01 via cms904usr.
source /opt/Xilinx/setup_xilinx.sh
xmd
XMD% connect mb mdm -debugdevice deviceNr 2 <-- we want to connect to the frontend, the second device
JTAG chain configuration
--------------------------------------------------
Device ID Code IR Length Part Name
1 042a2093 10 XC6VHX250T
2 442a8093 10 XC6VHX380T
MicroBlaze Processor Configuration :
-------------------------------------
Version............................8.40.b
Optimization.......................Performance
Interconnect.......................AXI-LE
MMU Type...........................No_MMU
No of PC Breakpoints...............2
No of Read Addr/Data Watchpoints...0
No of Write Addr/Data Watchpoints..0
Instruction Cache Support..........on
Instruction Cache Base Address.....0xf0000000
Instruction Cache High Address.....0xffffffff
Data Cache Support.................on
Data Cache Base Address............0xf0000000
Data Cache High Address............0xffffffff
Exceptions Support................on
FPU Support.......................off
Hard Divider Support...............off
Hard Multiplier Support............on - (Mul32)
Barrel Shifter Support.............off
MSR clr/set Instruction Support....on
Compare Instruction Support........on
Data Cache Write-back Support......off
Fault Tolerance Support............off
Stack Protection Support...........off
You should see something like: NB device 2 is a 380T, the front end.
See sub-directories in: cms904usr:~pamc/trigger_code/newest_bitfiles/ctp6