Normally it would on first start have a completely empty profile folder which would increase loading time and present Firefox as if it was your first time using it. So I made the following modifications:
added latest Flash plugin "FirefoxPortable\Data\plugins\NPSWF32.dll"
added an empty bookmarks file "FirefoxPortable\Data\profile\bookmarks.html"
set homepage to google, disabled phishing filter, and welcome tab "FirefoxPortable\Data\profile\prefs.js"
hide personal toolbar "FirefoxPortable\Data\profile\localstore.rdf"
I set the nsis launcher ini to hide the splash screen but it seems to force it in PE.