MICRO CABIN MUSIC PLAYER - Technical notes

Overview

1. tools/extract_disks.py extracts the songs (MCM) from the disk images and writes the playlist player/playlist.txt from the database tools/mcm_db.csv.
2. player/build_rom.py puts the songs, titles and pictures of the playlist, together with the game drivers and tables (instruments etc.) taken from the disk images, into an ASCII16 MegaROM.
3. The player in the ROM (player/player.asm) copies the driver of each song's game to RAM and plays the song with it.

Extracting songs (extract_disks.py)

• Each folder under disk_image is one game. Disks are numbered 1, 2, ... in file name order.
• Every position of every disk is checked for MCM data (tools/mcm_scan.py). MCM format: the first word is the number of channel blocks, then each block starts with the offset of the next block, ending with 0000. The 00 ending each channel's command list must be exactly at the end of its block (tools/mcm.py).
• Only songs using the FM chip and the PSG songs registered in the database are extracted. Songs excluded in the database (sound effects etc.) are skipped (--all extracts everything).
• File names are the first 4 letters of the folder name + disk number (1 digit) + serial (2 digits), e.g. ILLU102. FRAY_M2 / FRAY_TR use FRYM2 / FRYTR. PSG-only songs have P before the disk number and songs converted from Xak's bonus game have O (e.g. XAKP101, XAKO301).
• The title music of Xak's bonus game "GoGo Pixie" is converted to MCM from the PLAY / SOUND statements of OMAKE.BAS on the disk (tools/omake.py).
• A song used by several games is written to the folder of each game that has a row for it in the database.
• Finally the playlist and extract_log.txt (files vs. database rows) are written. If a playlist already exists, the new one is written to playlist_new.txt.

Database (mcm_db.csv)

A UTF-8 CSV (with BOM). One row is "one song of one game". Songs are identified by their content (MD5), so the same data with a different file name or disk position is the same song. When the same data is used by several games, each game has its own row and its own title (e.g. the same data is "Forest of Boroz" in Xak II and "Forest" in Tower of Gazzel).

Maintenance commands (tools/mcm_db.py):

python tools/mcm_db.py sync          # titles etc. of a full-format playlist.txt -> database
python tools/mcm_db.py playlist      # database + data -> playlist.txt
python tools/mcm_db.py find FILE     # which song a file is (also lists similar songs by channel)

extract_disks.py --full writes the full-format playlist (with title columns).

Playlist format

# comment
@startup ../data/CABIN/CABI112.MCM | タイトル | Title   # song at boot (not in the list)
@startup_cg ../data/cabin.png                          # picture at boot
@game Xak | Xak                                        # game name | English game name
@cg ../data/xak.png                                    # picture of the game
../data/XAK/XAK201.MCM                           # title from the database
../data/XAK/XAK202.MCM | 教会                     # title
../data/XAK/XAK202.MCM | 教会 |  |  | Church      # 5th field: English title
@song_cg ../data/xak_botu.png                    # picture only while the previous song plays
../data/ILLUSION/ILLU110.MCM | City | 1          # loop count
../data/FFANTASY/FFAN101.MCM |      | 0          # 0 = loop until the song ends or the time limit

• Paths are relative to the playlist file.
• Without a title, the database title is used ("Track nn" if not in the database). Without a loop count, the database value or --loops (default 2).
• The 4th field (the old driver selection) is ignored.
• English titles and game names are shown when the machine's character set (BIOS 002Bh) is not Japanese (Japanese if there is no English one). The byte at ROM address 001Eh (after the marker MCPLAYER LANG:) fixes the language: 00 auto, 01 English, 02 Japanese (or --lang when building).
• Letters, symbols, kana and kanji (JIS level 1 and 2) can be used. Characters not in the font become ? with a warning. Text after # is a comment, so # cannot be used in titles.
• A line is 256 dots (about 42 letters, 32 full-width characters); longer text is cut.
• Up to 999 songs. Identical song data is stored once. Up to 24 @song_cg.

Drivers (per game)

Each song is played with the sound driver of the game it comes from. When building the ROM, the driver is taken from the disk images in the disk_image folder with the same name as the song file's folder, patched and stored in the ROM. Before playing a song, the player copies its driver to its original address in RAM.

• Only the versions in drivers.json are used. It contains only addresses and MD5 values to identify each version, not the driver code. Candidates are found by the driver's entry code (jp interrupt / di, push ix ...) or by sig (the first bytes of the driver); the version is identified by the MD5 of the first 600h bytes and checked by the MD5 of the whole driver.
• A song uses the driver on the disk where the song was found, or else the driver on another disk of the same folder. When a disk has two versions (one for turboR etc.), the one in drivers.json is used.
• The addresses were found by comparing each driver with Roberto Vargas' disassembly of Illusion City's driver (mcdrv.asm, published on Manuel Pazos' website).
• Command numbers differ by driver (init 3, fade out 4, playing? 6 / 5, play 9 / 2). The channel work size (40h / 2Eh / 2Dh), loop counter offset etc. also come from drivers.json.

Patches applied when building the ROM (Drivers in build_rom.py):

• ld a,(ix+n) at the end of a loop (reading the loop address) -> call HookLoop (counts infinite loops)
• ld c,(ix+4) in the note routine -> call HookNote (tells the level meter about new notes)
• The 4 table pointers -> the address where the tables are placed
• Drivers with a fixed song address (Xak, Xak II, Gazzel, Fray, Final Fantasy): ld hl,C000h etc. -> A000h
• Illusion City, Fray (turboR): the FM write routine waiting on the turboR timer (E6h) is replaced by one that does not wait, and the wait-only routine by ret (turboR runs in Z80 mode)
• Final Fantasy: the ei at the command exit -> nop

Final Fantasy's driver (page 1)

FF's driver is part of the game program (from 4000h) and runs in page 1. Page 1 is this ROM, so the driver is placed in the RAM behind page 1 (the RAM in the same slot as page 3), and page 1 is switched to RAM only while the driver is called (with interrupts disabled, A8h and the sub-slot register are switched and restored afterwards). Therefore the hooks called by the drivers (HookLoop, HookNote) and the switching code are in page 3 RAM (RamHooks).

• The channel work is 2Dh bytes, and FM channels 7-9 are always rhythm (D_RHY = 0FFh).
• The work area starts at BD00h in page 2, so a song can be at most 7424 bytes.
• On machines without RAM in page 1, FF songs are skipped.
• If the FF disk has no driver, the Xak driver (or Xak II etc.) is used (DRV_FALLBACK).

Tables per game (instruments etc.)

The tables for envelopes, vibrato, effects and FM instruments (32 soft instruments), 0x1E0 bytes, differ per game. They are collected from the disk images when building the ROM, and the song's table is copied to page 2 RAM before playing it.

• On the disks the tables are stored in the same sequence as the songs and are used by the songs following them (on the Cabin Times disk, Xak II songs follow the Xak II table and Fray songs follow the Fray table). So each song is searched in the disk images and the nearest table before it on the same disk is used.
• The table address keeps the low 12 bits of its address in the game and is moved to 9000h-9FFFh (8F00h for Gazzel, which does not fit). With the same low bits, notes that read past the tables (the 37th note etc.) behave as in the game.
• If no table is found, the Illusion City (or other) table is used.
• The drivers and tables used are printed when building the ROM.
• The .fmins files of Manuel Pazos' PCPLAYER for Windows (FANTASY / FRAY / MCDRV (= Illusion City) / PMAKER / XAK) match the FM instrument part of the tables on the disks.

Behaviour

• At boot the player first checks that the mapper is ASCII16: a small routine copied to RAM selects bank 1 at 6000h and checks that the bank 1 marker (MC16, written by build_rom.py) appears at 4000h. If not, it prints a message with the BIOS and halts in RAM with interrupts disabled. With ASCII8 etc. the right part of the ROM is not visible at 6000h-7FFFh and calling code there would crash, so the code up to this point is in the first 8KB (4000h-5FFFh).
• The driver's interrupt routine is called from the H.TIMI hook, so slow screen updates do not disturb the tempo. Keys are also read every frame in the interrupt.
• End of a song
  • When every channel has repeated its infinite loop the given number of times, the song fades out and the next one starts (counted by HookLoop).
  • Songs without any infinite loop are detected when building the ROM; their loop count is set to 1 and the next song starts when they end (LOOP 01/01).
  • Songs that wait in a silent infinite loop (only rests) after the music (Xak's "Get item" etc.) are cut at that loop when building the ROM and treated as non-looping songs.
  • After 10 minutes the song is faded out (MAX_SECONDS).
• When a song has ended by itself, the sound chips are not reset before the next song (cutting the FM release causes a click).
• Channels after the 12th are not played.
• On PAL (50Hz) machines the driver is called once more every 5 interrupts, so songs play at the NTSC speed (60 per second). 50Hz is detected from the VDP setting (R#9) on MSX2 or later, and from the machine ID on MSX1.
• Sound chips: built-in MSX-MUSIC and FM-PAC are detected (FM-PAC I/O is enabled through 7FF6h). Without them only the PSG part of the FM song data plays (the separate PSG song data the games use without FM is not used). turboR is switched to Z80 mode at boot.
• At boot
  • MSX2 or later: the startup picture is shown and the startup song, if any, loops (silent if none). Any key stops the song and goes to the list. The H key goes to the list and opens the help without stopping the song, which then stops at its next loop point.
  • MSX1 (and with a V9990): the list is shown and the startup song plays once.
  • --autoplay: the startup song plays once, then the list plays from the first song (if there is no startup song, after showing the picture for about 3 seconds). Bit 0 of byte +2 of the playlist table.
• Play modes (E / R keys) and what happens when a song ends:

  	REPEAT	ONCE
  ALL	Next song; back to the first after the last	Next song; stop after the last
  GAME	Next song of the game; back to its first	Stop after the game's last song
  ONE	Same song again	Stop

• V key (MSX2 or later): switches between the picture of the playing game and the list. While a picture is shown, the bottom line shows "game / title" of the playing song, or the selected song when moving with the cursor keys.
• V9990 (GFX9000 etc.): if found at boot, pictures are always shown on the V9990 only, and the built-in VDP shows only the list (the V key is not used; pictures are shown even on MSX1). SCREEN 5 pictures use B1 (256x212) BP4, SCREEN 7 uses B3 (512x212, 512x424 for interlace) BP4, SCREEN 8 uses B1 BD8. Detection: after a soft reset through port 67h, a value written to R#15 can be read back.
• VDP at I/O 88h (V9968 cartridge, V9938 / V9958 upgrade kits etc.): checked when there is no V9990, and used like the V9990 (pictures are shown only there; HasExtVdp = 2). It is used as a V9958; the V9968 extension registers (R#20 / R#21) are not touched. SCREEN 5 / 7 / 8 use the same modes and VRAM layout as on the built-in VDP; interlaced pictures alternate SCREEN 7 page 0 (even lines) and page 1 (odd lines) (R#9 = 8Ch, R#2 = 3Fh). Interrupts are not used. Detection: the ID in S#1 is V9938 (0) or V9958 (2), and 55h, AAh written to VRAM 0 can be read back. Not checked when the built-in VDP itself is at 88h (BIOS 0006h = 88h).
• Built-in V9968 as the main VDP (I/O 98h): the type of the built-in VDP is not checked, so it is handled like a V9958 MSX2+. With a V9990 as well, pictures are shown only on the V9990, as above.

Pictures (CG)

• Files: images Pillow can read (PNG, JPEG, BMP, GIF ...) and BSAVE .SC5 / .SC8. Transparent parts become black.
• Size
  • 256x212: as is. 256x192 / 512x192 are padded with black at the bottom.
  • 512x212: a 512-dot screen, SCREEN 7 (16 colours).
  • 320x240 / 640x480: treated as emulator screenshots and the border is cut off (14 lines top and bottom for 212 lines, 24 for 192 lines, 32 dots left and right). 640x480 without doubled dots is a 512-dot screen, and with different odd/even lines an interlaced screen (512x424, two SCREEN 7 pages shown alternately).
  • Other sizes are resized to 256x212 (aspect ratio is not kept; a warning is printed).
• Colours
  • 256 dots wide: if it has 16 colours or less after conversion to the MSX2 512 colours (3 bits per RGB), SCREEN 5 (exact colours). Otherwise SCREEN 8 (fixed 256 colours) with error diffusion.
  • 512 dots wide: reduced to 16 colours if needed.
  • The text on the bottom line is white on black. If a SCREEN 5 / 7 picture uses all 16 colours, its brightest and darkest colours are used.
• Emulator screenshots (openMSX screenshot -raw) work best. Games write the palette directly to the VDP, so saved VRAM may not have the right palette. Images with scanlines or blur get more colours and become SCREEN 8.
• Compression: pictures are stored as "32-byte palette + run-length compressed screen data" and decompressed while sending them to the VDP / V9990 (cgrle.py: 01h-7Fh = n literal bytes, 80h+n = one byte repeated n+3 times). Runs never cross 256-byte boundaries or ROM bank boundaries, so decompression works 256 bytes at a time (letting interrupts through) and literals are sent with OTIR. Photo-like pictures compress poorly and take longer to show (about 1 second for a SCREEN 7 picture, about 1.9 seconds for a photo-like interlaced picture).

Screen

Text is drawn on SCREEN 2 (256x192) with k8x12 (8x12 dots) for kanji and kana and the full-width letters of k6x8 (6x8 dots, placed 2 dots below the top of a 12-dot line) for letters. " ' - ~ have no full-width glyphs, so similar characters are used. Characters are positioned in 2-dot units; half-width letters advance 6 dots, full-width 8 dots. The name table is filled with 0-255 so the screen is a bitmap, without a limit on the number of different kanji. A line is 12 dots high and the screen has 16 lines.

• A meter jumps to the note's volume when a new note starts and then falls one step every 3 frames; while the note sounds it stays at half its volume, and falls to 0 at rests. New notes come from HookNote; volumes come from the PSG registers (R7, R8-R10) and from the driver's channel work for FM (key on and volume).
• Used channels are decided when a song starts by checking whether each channel has notes (on PSG channels the instrument commands 42h-61h and FCh also make noise, so they count).
• The H key shows a help window (version, credits, keys) over the list.

ROM layout (ASCII16)

A playlist entry (12 bytes): bank / address / length of the song data, bank / address of the titles, loop count, driver/table set number (from 1 in SetDir), address of the game information.

Memory

• Drivers are made for a fixed address in the game and cannot be moved, so they are placed at their original address. When a song needs a different driver, its code, tables and initial values are copied from the ROM and its work area is cleared.
• If HIMEM is below DE00h, the system work area above it overlaps the drivers, so the original H.TIMI hook (disk timer) is not called.
• Page 2 must be RAM, so 32KB of RAM is needed (64KB for FF songs).

Tested

• With openMSX on turboR (FS-A1ST) and C-BIOS MSX2+ / MSX1: playing, end detection and loop counting with all 10 driver versions.
• All 255 songs played through and compared with the previous version (a ROM version of the Illusion City driver disassembly mcdrv.asm, and the PMAKER driver): the end times of 247 songs are the same. The differences are 3 Xak songs (XAK210, XAK301, XAK306: 2-7 seconds longer) and RISK202 of Dai Senryaku II (it stopped in the middle with MCDRV and now loops). Recorded sound is almost the same.
• Auto play without keys, V9990 display, a V9958 at I/O 88h (an openMSX extension; VRAM and palette match the data for SCREEN 5 / 7 / 8 and interlace; also checked on a real turboR with a V9968 cartridge), interlaced pictures, PAL speed, English display.
• Mapper check: with openMSX (FS-A1ST, C-BIOS MSX1) and a real ESERAMair set to 8KB banks, the player shows the message and stops; with ASCII16 it works on a real turboR with a V9968.
• The sound itself has not been fully checked by ear.

Fonts

"k8x12" and "k6x8" (letters) by Num Kadoma. Both are free to use, modify and redistribute (see font/misaki_12x8/k8x12.txt and font/k6x8/k6x8.txt).