Distributed filesystem in the GFS (assuming you mean RedHat's GFS2) sense, like GFS itself, SGI's CXFS, Apple's Xsan or even clustering support in almost all Enterprise DBMS's, does not mean that the data itself is distributed, only that the filesystem (or DB) logic is distributed and only shared component is single central (eg. FC or iSCSI attached) dumb block device without any special understanding of the filesystem, which is exact opposite of using local hard drives of the nodes.
Things like Lustre or GoogleFS use distributed storage nodes that are normally separate from client nodes.
While in both cases nothing prevents you from running both client applications and storage server process on same node there is little reason to do so, for both operational (disks are in one place) and economic reasons (attaching 20 disks to one node is cheaper than attaching 20 disks to 20 nodes).
In fact few years ago I've actively searched for filesystem with built-in redundancy and storage capacity distributed across client nodes and found almost nothing that was designed for such use case.
Only thing I'm aware of that was close to what you propose is/was essentially an wrapper for NFS in (open)MOSIX, that was able to redirect I/O from process to local disk when it found out that it was migrated to same node as NFS server runs on. With the important note that (open)MOSIX is not exactly meaningful HPC solution, but interesting nevertheless.
Things like Lustre or GoogleFS use distributed storage nodes that are normally separate from client nodes.
While in both cases nothing prevents you from running both client applications and storage server process on same node there is little reason to do so, for both operational (disks are in one place) and economic reasons (attaching 20 disks to one node is cheaper than attaching 20 disks to 20 nodes).
In fact few years ago I've actively searched for filesystem with built-in redundancy and storage capacity distributed across client nodes and found almost nothing that was designed for such use case.
Only thing I'm aware of that was close to what you propose is/was essentially an wrapper for NFS in (open)MOSIX, that was able to redirect I/O from process to local disk when it found out that it was migrated to same node as NFS server runs on. With the important note that (open)MOSIX is not exactly meaningful HPC solution, but interesting nevertheless.