To write a device driver you need to first know about how the device works. If you have documentation for the N64 GPU then you're already in good hands. What you'd first want to do is read that documentation, and see how you could programmatically create an interface that can send, receive, measure, read, write, etc. A good example you can start with (and that's probably much easier) is a PS/2 mouse device driver. You'll send bytes, track the positional-axis that the mouse trackball/left/right click sends from its microchip to the port 0x60 or the like (if set up to do so; check the link above for more info). After that, you need to structure this input within the scope of a program that can make adequate calls to read/send information. This is, in general, the same way with any piece of hardware, but much more difficult with N64's GPU.
Another similar, and possibly even easier method is the equivalent PS/2 keyboard, which sends scan codes that are read, and simple flow-of-logic code can decide what to do with the bytes it receives. If you're working at a low-level like this you're probably familiar with assembly languages for various architectures, as these will almost always be necessary in some cases. It's a bit intense, but since you mentioned the N64 GPU I'll do my best to add this:
To render graphics, the N64 CPU constructs a display list (GBI command list) and designates the RSP as the processor to use to render graphics. The RSP outputs the resulting graphic drawing into the frame buffer in RDRAM by simply reading and executing the display list provided by the CPU. The accumulated content of the frame buffer is output as a video signal from the video DAC (digital-to-analog converter) by way of the VI (video interface). The following diagram summarizes this process(check N64's Wiki for all of the abbreviations mentioned):

You'd probably want to operate through DMA, and you should be aware of the memory map for the N64.
DMA registers:
$A460:0000 = RAM address (address & 0x00FFFFFF) $A460:0004 = ROM address (address & 0x1FFFFFFF) $A460:0008 = Transfer size (from RAM to cartridge) $A460:000C = Transfer size (from cartridge to RAM) $A460:0010 = DMA Status
And here's the entire memory map:
0x0000 0000 to 0x03EF FFFF RDRAM Memory
0x03F0 0000 to 0x03FF FFFF RDRAM Registers
0x0400 0000 to 0x040F FFFF SP Registers
0x0410 0000 to 0x041F FFFF DP Command Registers
0x0420 0000 to 0x042F FFFF DP Span Registers
0x0430 0000 to 0x043F FFFF MIPS Interface (MI) Registers
0x0440 0000 to 0x044F FFFF Video Interface (VI) Registers
0x0450 0000 to 0x045F FFFF Audio Interface (AI) Registers
0x0460 0000 to 0x046F FFFF Peripheral Interface (PI) Registers
0x0470 0000 to 0x047F FFFF RDRAM Interface (RI) Registers
0x0480 0000 to 0x048F FFFF Serial Interface (SI) Registers
0x0490 0000 to 0x04FF FFFF Unused
0x0500 0000 to 0x05FF FFFF Cartridge Domain 2 Address 1
0x0600 0000 to 0x07FF FFFF Cartridge Domain 1 Address 1
0x0800 0000 to 0x0FFF FFFF Cartridge Domain 2 Address 2
0x1000 0000 to 0x1FBF FFFF Cartridge Domain 1 Address 2
0x1FC0 0000 to 0x1FC0 07BF PIF Boot ROM
0x1FC0 07C0 to 0x1FC0 07FF PIF RAM
0x1FC0 0800 to 0x1FCF FFFF Reserved
0x1FD0 0000 to 0x7FFF FFFF Cartridge Domain 1 Address 3
0x8000 0000 to 0xFFFF FFFF External SysAD Device
Some more info for further reading (even though it may not 100% help):
RSP(Reality Signal Processor)
RSP is your transform and lighting unit (TnL). Manipulates world data and textures.
Mathematically, lots of matrices to transform from local data -> world space -> view space (projection w/ z-perspective correction).
Transform means scaling, translating and rotating for polygons, lighting normals, texture UVs.
Creates primitive lists of triangles and lines for the RDP to render.
RDP(Reality Drawing Processor)
RDP is the display unit.
Rasterizer, fog, environmental, color blending. Anti-aliasing effects.
Lower-level pixel handler.
uCodes
The RCP has its own language, dubbed 'uCodes' (256 microcodes).
Think of modern vertex and pixel DirectX shaders - both stages combined.
R4000 coprocessor (COP2). Each uCode is a string of ASM instructions run by the RSP. Also sets up the RDP batch renderer.
Display lists are a sequence of uCodes defined by the game. This is fed to the RSP.
Note: Emulator authors choose to translate uCodes into higher-level languages.
The programmers can define their own vertex / texture formats. Lighting methods. Overdraw detection and other flexible wizardry.
The microcodes are uploaded to the RCP at run-time. So each game has its own library of drawing functions (some are like DSP1 -> DSP1A -> DSP1B and others are akin to DSP2,DSP3).
But I'll just go all out here for the N64:
Main 2D Drawing Routine
The main drawing routine forms the executing core for the 2D image drawing sample. The process is completed in three steps that are executed over and over in a loop:
Construct the display list in the CPU.
Transfer the display list to the RCP for execution. The RCP creates the actual display data and outputs the video signal by way of the frame buffer, VI, and video DAC as previously outlined in this chapter.
Keep the CPU and RCP processes in sync, and return to step 1.
The following code points to the display list, sets the sprite size, and sets up the loop(all psuedocode):
void entry(void)
{
Gfx *gp; /* points to display list */
u16 w, h;
w=64; h=64; /* sets up sprite size */
while(1){
Step 1 -- Construct the display list
The following code reserves the necessary memory area for the construction of the display list, sets up the RCP execution process for sprite drawing in the reserved area, and provides a termination process for the constructed display list. Each code snippet is followed by s short explanation.
/* Start to construct a display list */
gp = gfxBegin(1024);
This code checks to see if a display list has already been constructed. If a display list hasn't been completed, this code reserves the GBI command area for new construction. If it has been completed, this code moves on to step 2, the process that transfers the display list to the RCP.
/* Set the drawing mode for RSP and RDP */
gp = setup_SP_DP(gp);
This code constructs a command that sets the necessary RCP drawing mode, and then adds that command to the display list.
/* Accept the texture pattern */
gp = load_texture(gp,w,h);
This code sets up a texture pattern loading command in the display list.
/* Write the texture pattern */
gp = draw_texture(gp,124,92,w,h);
This code sets up a texture drawing command in the display list.
/* End the construction of display list */
gfxEnd(gp);
This code terminates the display list.
Caution: Watch out for an unterminated display list. If it is transferred to the RCP, it will cause the RCP to hang (stop responding). Also, be sure to put the gDPFullSync function at the end of each display list. Otherwise, the RDP end message won't ever come.
Step 2 -- Transfer the display list to the RCP to execute the drawing process
The following code transfers the display list to the RCP where the display list is interpreted and executed.
/* Transfer display list to RCP */
gfxFlush( );
This code transfers the display list to the RCP where it is executed. This function also provides the frame buffer switch that writes the created image data into the frame buffer.
Step 3 -- Synchronize the CPU with the RCP
The following function ensures coordination between the CPU and RSP:
/* Wait for the retrace */
gfxWaitSync( );
}
}
Techniques for Construction of the Display List
The actual construction of the display list uses one of these processes:
Set the RSP and RDP Drawing Modes
Set and Read the Texture
Set the Drawing Sequence of the Bitmap Pattern
Set the RSP and RDP Drawing Modes
The following routines set the RCP drawing mode that actually creates the commands that render the drawing reflected in the display list.
The gSP and gDP functions are included in the N64 library; for more information about them, please see the online N64 Function Reference Manual (HTML manual pages).
static Gfx *setup_SP_DP(Gfx *gp)
{
/* Set all sorts */
/* Set the texturing parameter */
gSPTexture(gp++,0x8000,0x8000,0,
G_TX_RENDERTILE,G_ON);
/* The synchronous setting between the
rendering and the sub-attribute */
gDPPipeSync(gp++);
/* Set the RDP cycle type */
gDPSetCycleType(gp++,G_CYC_COPY);
/* Set the rendering mode of the
blender within RDP */
gDPSetRenderMode(gp++,G_RM_NOOP,G_RM_NOOP2 );
/* Set the texture LOD */
gDPSetTextureLOD(gp++,G_TL_TILE);
/* Set the perspective of the texture map */
gDPSetTexturePersp(gp++,G_TP_NONE);
/* Set the detail type */
gDPSetTextureDetail(gp++,G_TD_CLAMP);
/* Set the texture filter type */
gDPSetTextureFilter(gp++,G_TF_BILERP);
/* Set the conversion mode of
the color space */
gDPSetTextureConvert(gp++,G_TC_FILT);
/* Set the compare mode of the alpha value */
gDPSetAlphaCompare(gp++,G_AC_NONE);
/* Set the dithering mode of the color data */
gDPSetColorDither(gp++,G_CD_DISABLE);
/* Set the dithering mode of the alpha value */
gDPSetAlphaDither(gp++,G_AD_NOISE);
return gp;
}
Set and Read the Texture
The gDP functions are included in the N64 library; for more information about them, please see the online N64 Function Reference Manual (HTML manual pages).
static Gfx *load_texture(Gfx *gp,u16 w,u16 h)
{
/* Read TLUT */
/* Set the texture look-up table */
gDPSetTextureLUT(gp++,G_TT_RGBA16);
/* Read the texture look-up table */
gDPLoadTLUT_pal16(gp++,0,texturetlut);
/* Read the bitmap pattern */
gDPLoadTextureTile_4b(gp++,texture,
G_IM_FMT_CI,w,h,
0,0,w-1,h-1,0,
G_TX_WRAP | G_TX_NOMIRROR,G_TX_WRAP |
G_TX_NOMIRROR,
G_TX_NOMASK,G_TX_NOMASK,G_TX_NOLOD,
G_TX_NOLOD);
return gp;
}
Set the Drawing Sequence for a Bitmap Pattern
The following code sets the drawing sequence for an accepted texture image. The gSP functions are included in the N64 library; for more information about them, please see the online N64 Function Reference Manual(HTML manual pages).
static Gfx *draw_texture(Gfx *gp,
u16 left,u16 top,u16 w,u16 h)
{
/* The bitmap pattern drawing */
gSPTextureRectangle(gp++,
left<<2,top<<2,((left+w)<<2)-1,
((top+h)<<2)-1,
G_TX_RENDERTILE,
0,0,4<<10,1<<10);
return gp;
}