Eric Holscher, one of the creators of Read The Docs, recently posted about the importance of a documentation culture in Open Source development, and about things that could be done to encourage this. He makes some good points, and Read The Docs is a very nice looking showcase for documentation. Writing good documentation is difficult enough at the best of times, and one practical problem that I face when working on Sphinx documentation is that I often feel I have to break away from composing it to building it, to see how it looks - because the look of it on the page will determine how I want to refine it.
What I’ve tended to do is work iteratively by making some changes to the ReST sources, invoking make html and refreshing the browser to show how the built documentation looks. This is OK, but does break the flow more than a little (for me, anyway, but I can’t believe I’m the only one).
I had the idea that it would be nice to streamline the process somewhat, so that all I would need to do is to save the changed ReST source – the building and browser refresh would be automatically done, and if I had the editor and browser windows open next to each other in tiled fashion, I could achieve a sort of WYSIWYG effect with the changes appearing in the browser a second or two after I saved any changes.
I decided to experiment with this idea, and needed a browser which I could easily control (to get it to refresh on-demand). I decided to use Roberto Alsina’s 128-line browser, which is based on QtWebKit and PyQt. Roberto posted his browser code almost a year ago, and I knew I’d find a use for it one day :-)
I also needed to track changes to .rst files in the documentation tree, and since I do a fair amount of my Open Source development on Linux, I decided to use inotify functionality. Although there is a Python binding for this, I decided to use the command-line interface and the subprocess module in the standard library, because I wasn’t very familiar with inotify and the command-line interface is easier to experiment with.
The basic mechanism of the solution is that the browser watches for changes in source files in the documentation tree, invokes Sphinx to build the documentation, and then refreshes its contents. This is done in a separate thread:
class Watcher(QtCore.QThread):
def run(self):
self._stop = False
watch_command = 'inotifywait -rq -e close_write --exclude \'"*.html"\' .'.split()
make_command = 'make html'.split()
while not self._stop:
# Perhaps should put notifier access in a mutex - not bothering yet
self.notifier = subprocess.Popen(watch_command)
self.notifier.wait()
if self._stop:
break
subprocess.call(make_command)
# Refresh the UI ...
self.parent().changed.emit()
def stop(self):
self._stop = True
# Perhaps should put notifier access in a mutex - not bothering for now
if self.notifier.poll() is None: # not yet terminated ...
self.notifier.terminate()
The thread invokes inotifywait, and waits for it to exit. This happens when a file is written in the documentation tree which has an extension other than .html, and this typically happens when a source file is edited and saved. The inotifywait command is usually available through a Linux package – on Ubuntu, for example, you can install it using sudo apt-get install inotify-tools. In the specific invocation used, the –r flag tells the program to recursively watch a particular directory, –q indicates that output should not be too verbose (it’s used for trouble-shooting only), –e close_write indicates that we’re only interested in files being closed after being opened for writing, and --exclude '"*.html"' indicates that we don’t care about writes to .html files.
The Watcher instance’s parent is the main (browser) window. In this we create a new custom event, changed, to be emitted when we want the window to know that the HTML has changed. This is done through the following snippets of code:
changed = QtCore.pyqtSignal()
which is declared in the main window class, and
self.watcher = Watcher(self)
self.changed.connect(self.wb.reload)
self.watcher.start()
which are added to the window’s constructor. Here, self.wb is the QWebView component of the browser which actually holds the browser content.
One last refinement is to save the browser window coordinates on exit and restore them on starting, so that if you have moved the window to a particular location, it will reappear there every time until you move it. First, we create a module-level QSettings instance:
settings = QtCore.QSettings("Vinay Sajip", "DocWatch")
and provide a couple of main window methods to load and save the settings:
def load_settings(self):
settings.beginGroup('mainwindow')
pos = settings.value('pos')
size = settings.value('size')
if isinstance(pos, QtCore.QPoint):
self.move(pos)
if isinstance(size, QtCore.QSize):
self.resize(size)
settings.endGroup()
def save_settings(self):
settings.beginGroup('mainwindow')
settings.setValue('pos', self.pos())
settings.setValue('size', self.size())
settings.endGroup()
When the main window is closed, we need to stop the watcher and save the settings. (We also need to call load_settings in the main window constructor.)
def closeEvent(self, event):
self.save_settings()
self.watcher.stop()
The last thing is to construct the code which is invoked when the module is invoked as a script. Note that this very simplistic use is consistent with Sphinx’s quick-start script defaults.
if __name__ == "__main__":
if not os.path.isdir('_build'):
# very simplistic sanity check. Works for me, as I generally use
# sphinx-quickstart defaults
print('You must run this application from a Sphinx directory containing _build')
rc = 1
else:
app=QtGui.QApplication(sys.argv)
path = os.path.join('_build', 'html', 'index.html')
url = 'file:///' + pathname2url(os.path.abspath(path))
url = QtCore.QUrl(url)
wb=MainWindow(url)
wb.show()
rc = app.exec_()
sys.exit(rc)
The code (MIT licensed) is available from here. As it’s a single file standalone script, I haven’t considered putting it on PyPI – it’s probably easier to download it to a $HOME/bin or similar location, then you can invoke it in the docs directory of your project, run your editor, position the browser and editor windows suitably, and you’re ready to go! Here’s a screen-shot using doc-watch and gedit:

Please feel free to try it. Comments and suggestions are welcome.
Update: Another advantage of using the subprocess / command line approach to notification is that it’s easy to slot in a solution for a platform which doesn’t support inotify. Alternatives are available for both Windows and Mac OS X. For example, on Windows, if you have IronPython installed, the following script could be used to provide the equivalent functionality to inotifywait (for this specific application):
import clr
import os
from System.IO import FileSystemWatcher, NotifyFilters
stop = False
def on_change(source, e):
global stop
if not e.Name.endswith('.html'):
stop = True
print('%s: %s, stop = %s' % (e.FullPath, e.ChangeType, stop))
watcher = FileSystemWatcher(os.getcwd())
watcher.NotifyFilter = NotifyFilters.LastWrite | NotifyFilters.FileName
watcher.EnableRaisingEvents = True
watcher.IncludeSubdirectories = True
watcher.Changed += on_change
watcher.Created += on_change
while not stop:
pass
Whereas for Mac OS X, if you install the MacFSEvents package, the following script could be used to provide the equivalent functionality to inotifywait (again, for this specific application):
#!/usr/bin/env python
import os
from fsevents import Observer, Stream
stop = False
def on_change(e):
global stop
path = e.name
if os.path.isfile(path):
if not path.endswith('.html'):
stop = True
print('%s: %s, stop = %s' % (e.name, e.mask, stop))
observer = Observer()
observer.start()
stream = Stream(on_change, os.getcwd(), file_events=True)
observer.schedule(stream)
try:
while not stop:
pass
finally:
observer.unschedule(stream)
observer.stop()
observer.join()
Thanks Vinay, This problem has been stopping me from upgrading to stretch and now I can go ahead.
ReplyDelete