63 replies to this topic

[Arvy]

A WinBuilder DBMS prototype has now reached the v0.10 beta stage in response to comments that were offered on the earlier WinBuilder Browser project. (See here and here .) It should be noted and emphasized at the outset that this development route DOES NOT CONTEMPLATE and WILL NOT REQUIRE any database installation whatever by any end user, NOR by any potential client developer. The database itself can reside and run anywhere and is, at least potentially, a distributable client-server system.

The intention is to provide all of the advantages of fully searchable* RDBMS-supported project management, tracking, versioning, cataloging, downloading, etc. across the entire spectrum of multi-client, multi-contributor WB development with the absolute minimum of manual inputs by any of the parties involved and without the need for local listing upkeep by developers who have better things to do. At the same time, to facilitate possible transition, it remains fully compatible with and updatable by WinBuilder's existing system of .INI files that are locally maintained for each WB project. (*Only rudimentary search capabilities are implemented to date, but the possibilities are virtually unlimited.)

The WinBuilder DBMS can update its own records either from directly acquired file data on a client site (a "one-touch" client command) or by fully automated importation of .INI file data from remote sites, regardless of whether the latter are active DBMS clients. In fact, with only minimal PHP set-up by participating client developers, it can also generate their local .INI files automatically so long as they may continue to be needed for any purpose.

Client set-ups require no "config file" editing at all. As will be evident when exploring the administrative interface, all client configuration and preference settings, including language and stylesheet selections, are handled by the DBMS itself and may be modified by the client's designated administrator as desired using "self-documented" online forms. The designated client administrator may also add, edit and delete additional users with specified privileges at will.

For additional information, see the online manual.

To view and test online, see the demo installation. (Admin login as "admin", password "admin")

To download your own DBMS beta client package, fill in the database registration form.

PLEASE NOTE: While every good faith effort will be made to protect and preseve all data that is entered by DBMS clients, it should be understood that, especially during the beta testing phase, it is impossible to guarantee total data integrity. Local INI files will, in any case, continue to be held in place by all sites thus providing their own fallback regardless of DBMS client installation and usage.

[Arvy]

UPDATE: Just a couple of early screenshots for those who are more motivated visually than by mere words. I suppose that's almost everybody.

User Interface - Search results:


Admin Interface - Client Preferences:

[pscEx]

UPDATE: Just a couple of early screenshots for those who are more motivated visually than by mere words. I suppose that's almost everybody.

User Interface - Search results:


Admin Interface - Client Preferences:

The first screenshot:
It really helps when looking for a certain script.
(May I steel your idea to build into downloadEx? )

To the second one I have some (WinBuilder system) concerns:

Currently there are a couple of .script developers. Each of them 'owns' one or more servers where projects / scripts can be stored by him / her.

But each of them also can only maintain his / her 'own' server.

I see some difficulties to make a 'generalized' admin access for all servers.

Peter

[Arvy]

Peter, there is almost NOTHING that cannot be accomodated and accomplished with an RDBMS-based sytem where each client establishes his own individual configuration parameters within the database itself. I've already tested its capabilities for data import, etc. with every one of the clients currently listed before changing the params to "fake" settings for demo purposes.

As for your "stealing" my ideas, any work that I have ever done for any community, mine, unlike some others, has always been provided as completely open-sourced upon completion. All of my file headers, when released, read as follows:

// ------------------------------------------------------------------------- //
// Original Author : Richard Virtue ("Arvy") - http://virtech.org/
// Licence Type : Public GNU/GPL
// ------------------------------------------------------------------------- //

But this one is, as yet, neither completed nor released. So let your own conscience be your guide.

[Arvy]

UPDATE #2: To facilitate client project management, a full-featured PHP file manager has been added to the "Client Operations" section of the administrative interface.



While its appearance is a bit ugly at this early stage (opens in its own window 'cuz it needs the working space), the file manager can accomodate Shell Execute commands and archive/unarchive functions (TAR, GZIP, BZIP2 and ZIP) as well as the usual upload, copy, move, rename and chmod operations.



Due to PHP limitations, most site managers will usually prefer FTP uploading for large files, but the PHP file manager is able to handle multi-file uploads when the file sizes are within PHP limits.



Please play around with it on the Test Client Server site and help me to find all the bugs before any beta release if we can. I'm sure it has a few at this early prototype stage.

__
SPECIAL COMMENT RE FILE ARCHIVING WITH PHP (Nuno Brito please note): For some reason (perhaps deliberate, perhaps not) the PHP set-up on the winbuilder.net server does not have the ZIP.DLL extension enabled. For this reason, no one on that sever can use PHP scripts to UNarchive ZIP-compressed files. All TAR, GZIP, BZIP2 and ZIP compression and all TAR, GZIP and BZIP2 decompression can be handled by the file manager's internal functions. But, like any similar PHP application, it needs the ZIP.DLL extension enabled to handle decompression of ZIP archives. It would seem like a very useful thing to have enabled on the winbulder.net server in any case.

[allanf]

Hi Arvy,

I have been playing with this. If anything is broken, I'll have to put my hands up.

I uploaded a little project 'pe2', then downloaded it with Winbuilder's Download Center.

The File Manager looks extremely powerful. When it first starts, it allows access to all areas from the navigation '<</home/arvy/browser.winbuilder.net/dbclient' in the right pane. After selecting a project directory from the left tree, the navigation seems to become restricted with ''/home/.fluke/arvy/browser.winbuilder.net/projects' (no '<<').

It is a bit scary to see an edit button next to the files of every project on the server - not just the test projects - when logged in so easily. ... and copy, move, delete, etc.

A small issue: When navigating in the 'User Page' - changing projects in the drop-down - it is necessary to click 'Go' twice to refresh the file list. (EDIT: Not to worry. I know why - the first click of 'Go' updates to 'Folders' drop-down.) Also the File Manager seems a little bit sticky in the same regard when changing project directories - although I couldn't get consistent results.

And even smaller issue: Some windows from the File Manager need resizing to show all the buttons. eg 'Permissions' Window, and the 'Edit' File Window.

Regards

[Arvy]

Hi Arvy,

I have been playing with this. If anything is broken, I'll have to put my hands up. ...

Hi allanf.

Well, it's not supposed to be broken, but I'm sure there are probably a few bugs to be found somewhere. They're usually discovered right after an initial release.

I'm not even sure if this thing will actually turn out to be useful to anyone or not. But, in the meantime, I'm having a little fun just discovering what is possible based on the assumptions that (1) developers must find local list keeping at least slightly annoying and (2) there may be some "community" advantages in a somewhat more organised and integrated approach to "cataloguing" the total WinBuilder output across all projects -- if not immediately, then at some point in the foreseeable future. Anyhow, all of your comments are greatly appreciated. Thanks very much.

I can well understand that the overall "exposure effect" may appear a bit scary, but most of that is simply inherent in the powerful nature of the combined PHP-SQL "beast". In many respects, the separate DBMS records actually add security. You can be assured that the secure integrity of other people's sites and work efforts (not to mention my own) is always uppermost in my mind to the extent that I will never accept any compromise in that area regardless of opportunity or inducement.

Yes, the demo test set-up is quite "wide open" as it must be to "display the goods", so to speak, but not without safeguards that may not be obvious. I've hobbled the demo a bit more to keep from scaring others too much, although I sometimes wonder if people really understand how vulnerable they are and how important security is. When it comes to thievery and just plain mean destructiveness, not everyone shares the same ethical standards. Anyhow, you can certainly expect some "safecracker security" in anything that is actually released, even at the beta stage. That's one of several reasons why there is no downloadable client package available yet.

The user "frontend" is definitely sloppy in many ways and you've put your finger on one that needs fixing right away. In fact, having spent most time to date focussed on "internals" and on client "self-administration" convenience, I intend making the user page my next priorirty. Hopefully, you'll be seeing some major improvements in the immediate future. (Think I've got that silly and annoying server-project-folder selection and display nonsense beat now if you have time for another quick look.) In any case, the user page definitely needs vast improvment as the first-seen "portal" to the rest of the package. As it stands, I don't think it could ever make a dime in the (self-)promotional and advertising game.

As for the client file manager, it's really just "tacked on" for the time being. Proper integration of its "framed" windows into the overall admin interface will take a little while, but should be accomplished prior to any beta release -- I hope.

Thanks again for taking the time to give me your very welcome feedback.

[Arvy]

UPDATE #3: Narrowing the gap toward availability of a downloadable beta package, an automated client registration process is now in place and fully functional.



Unfortunately, I can't offer this one for pre-testing on the open demo platform. If you click the «Register WinBuilder DBMS Client» link at the bottom of the admin login box, it will go through the motions but won't "deliver the goods". Its actual operation would have results that we're not quite ready for yet.



Any real clients that might get added, would introduce considerable difficulty into any major DBMS structural changes that might yet be required to accomodate overlooked data fields.

The next step is the one most developers hate most. Complete documentation. Aaarrrghhh!

[Arvy]

An online manual is now available. It's only preliminary, but I don't think I'm alone in saying that documentation is my least favourite development job. It will grow. I promise.

The new client registration feature remains temporarily unavailable at this time. It is anticipated that a version 0.10 beta package will be available shortly (7-10 days) at which time the "self serve" client registration will be enabled.

Client reqistration is being held in abeyance to ensure that all data that is entered thereafter can be fully preserved regardless of any dB structural changes that may be needed to accomodate feature requests in subsequent releases. The establishment of multiple offsite backup-restore facilities for that purpose (and simply as a matter of sound dB management practice) will require a few more days.

[Arvy]

WinBuilder DBMS v0.10beta client packages are now available by completing the database registration form. Please see the top posting in this thread for details and links to additional information.

Those registering as primary client site administrators should note that whatever privileges (Upload, Update, Delete, Admin) they may grant to any others they may choose to add currently apply to all projects on their respective client sites. It is anticipated the future versions will make such privileges assignable to specific projects, but that is not currently the case. It's "in the mill" and should require only a week at most.

[Arvy]

As previously noted, primary site administrators may choose to add secondary "assistant" logins for their client domains. It is now possible to assign any such secondary logins permissions for Upload, Revise, Delete and Compress on a per project basis. See the illustration below or try a secondary login (uname: "assistant", pwd: "assistant") on the demo site to experience the restriction effects first hand.

Other permissions, such as Admin (administration rights) and FileMan (file manager access), remain at the domain level as they only make sense at that level. Some permutations and combinations that are possible (e.g., file manager access combined with limited project upload permissions) are also quite nonsensical, but individual client site administators can choose whatever options suit their own needs as they perceive them.

[Arvy]

Hi Nuno. Thanks for the thank you button. I thought you were going to spend the weekend outdoors. You'll never get a tan hanging around here.

I'm not so sure about that "useful post" part. I strongly suspect that, just as with the earlier browser project, most, if not all, WB developers will prefer staying with what they've become accustomed to. But it's been a fun exercise for me regardless.

Just a few brief words of explanation for those who may wonder about all the WB project data that is already available for end users to browse. As mentioned in the manual, this application was designed to include full "backward compatibility" with the existing WB project system and most existing data has thus been imported by the host admin (me) for demo purposes. The intention (theoretical) would be to eliminate the imported data in favour of direct client domain updates when available.

[allanf]

Hi Arvy,

....... ...... HELP!!

I am having problems with the domain setup... I think ... ...

In the manual, I don't understand:

Install the downloaded client package in any (sub)folder on your site that has direct HTTP and FTP access to the absolute paths that you entered in the registration form

.

... but I'm not sure if that is my problem.

The Client is registered as 'pe2 Project', and the WebSite is http://www.pe2.winbuilder.net/

I can select the Client as a regular user from http://virtech.org/w...Host/wbuser.php
, and can log in as an administrator, but can't navigate to any files or folders in any case.

From what I can see, my server is similar to your 'browser.winbuilder.net' except I don't have '/projects/' as an absolute HTTP Path.

I'm happy to appoint you as an assistant if you think I need serious help. ... ...

Thanks

[Arvy]

All of your registered client domain settings appear to be correct for your type of set-up -- i.e., with your WB projects folder being the same as the HTTP root folder for your site.

Did you install the "customized" client package that your downloaded into a subfolder of your HTTP root? It is only by logging into that client package installed on your own site that you can actually perform the project browsing and file management operations that are included under the administrative interface Client Ops tab. Neither you nor anyone else can carry out "local" ops on your projects by logging into any other package that is installed anywhere else, regardless of whether it's another client site or the host site.

So, at the http://virtech.org/winbuilder/ site, even if you log in there, your will only see the data representing your projects as any ordinary user would. Logging in there (or anywhere other than your own site's client package) as the admin for your domain won't let you administer and maniplate your projects as if they were actually located on that other server.

As Mickeysoft would say: "It's not a bug; its a feature" and it's intended for your own protection. Not even host admins should have that kind of invasive power.

[allanf]

I can select the Client as a regular user from http://virtech.org/w...Host/wbuser.php
, and can log in as an administrator, but can't navigate to any files or folders in any case.

Arvy,

Immediately after posting the above, I tried again, and had success as a regular user...

But, after logging in (selecting 'pe2 Project' as the domain), I seem do be directed only to the WB Demo Client on your Website.



Thanks again

[Arvy]

Well, I've checked both your database registration entries and the your customized client package (the same one that you should have downloaded and installed on your site) and they are both correct in every detail.

For the moment, you have me baffled and I can only repeat my earlier questions about WHERE you installed your client package and WHERE you are now logging in. If the answer to either question is anywhere other than a (sub)folder that is located within your own client domain site, that would certainly account for your problem. Any installation or login location outside of your own site cannot possibly translate its absolute HTTP and FTP paths into the correct "local" working locations for PHP file management operations.

I'm also slightly puzzled by your comment about "selecting 'pe2 Project' as the domain". If you are using your own client package it should automatically pre-select the correct domain for you to log into simply by using the primary administrator name and password that you provided. You shouldn't need to select your domain from any list unless you are logging into the demo location's set-up instead of your own client package.

[allanf]

For the moment, you haver me baffled and I can only repeat my earlier questions about WHERE you installed your client package and WHERE you are now logging in....

OK! ... ...

Currently, the client package is installed in two locations '/' and '/pe2.winbuilder.net/'.

I was trying lots of different things last night, and nothing seemed to go right...

It must have been my last cup of coffee this morning that fixed it.

Thanks

[Arvy]

Heh heh. I'll drink to that -- but my coffee may include a good stiff shot of Navy Rum!

You had me a little worried there for a minute or two But all's well that ends well.



__
P.S.: Thanks for the thank you button. I didn't notice it until I happened to look again at the top post. If you folks keep telling me this thing is "useful", I might even begin to believe it.

__
P.P.S.: Just a few words to try to make as clear as possible the security/protection factors involved in using any of the "Client Ops" PHP file manipulation features that are included in this application. There are three (3) elements involved, all of which must be satisfied in order to use those features successfully:

1. The client registration information for absolute HTTP and FTP paths to the main projects folder that is entered into the database must correspond to the actual client site layout.
2. The DBMS must recognise the information that is contained in the client's customised download package as correctly identifying that particular client upon login via that particular package.
3. The downloaded client package must itself be installed in a (sub)folder that resides within the domain site that it identifies and be accessed there for logging in.

As stated above, these conditions and restrictions are there for your own protection and should not be regarded as "bugs" or nuisance factors. Amongst other things, they ensure, for example, that no mischievous person can simply enter "misappropriated" data claiming to be for your site and use the downloaded client package thus obtained to "administer" your site from some other location.

Notwithstanding all of the foregoing, it should be clearly understood that neither this nor any other add-on package can possibly overcome or compensate for security holes that may be present in the site itself and/or in the set-up of the server on which it is located. That responsibility is and must be entirely your own and your host server administrator's.

[allanf]

You had me a little worried there for a minute or two But all's well that ends well.

Well I think I managed to break it this time. ... ...

... with 'dot's in my project folder names, like 'pe21.001'. I hadn't clicked 'Update DB' or 'Create INI' before, but was trying to expose all files and folders on my server. The 'Update DB' generated a string of errors about directories (named without dots) not found.

What choices do I have to fix it?

The current updates.ini and index.html were created and uploaded by Winbuilder, and http://pe2.winbuilder.net looks OK, but DBMS is still looking for folders without dots.

Another thing that is not clear to me... why is my 'WBDBClient' folder exposed to the user interface when browsing? What I mean is, how can I hide it? It's not visible in your 'Test Client'.

As previously mentioned, I have the Client Package in two locations '/' and '/pe2.winbuilder.net/'. The latter seems to me to be the 'active' package - tested by renaming each one - and also the one that is exposed.

Don't be too concerned about it. Some people lead by example. I follow by example... a quck snapshot of the directory structure of your 'Test Client' and what buttons to press might do the trick.

BTW, I vaguely recall a discussion about the 'dot' issue... not sure if it related to your DBMS. I tried clicking the 'here' links in on the first post... they're not leading to any thread, just redirected to forum index?

Regards

[Arvy]

Well I think I managed to break it this time. ... ...

If so, that's good. Exposing any weaknesses is what beta testing is all about.

If you don't mind, I'm going to "reorganise" your questions just a bit because I think we may be able to paint a somewhat clearer picture by dealing with them in a slightly different order.

... why is my 'WBDBClient' folder exposed to the user interface when browsing? What I mean is, how can I hide it? It's not visible in your 'Test Client'.

Fundamentally, it's just a matter of how you choose to organise your own site. What goes into the database is whatever you include under (i.e., subordinate to) your registered main WB projects folder (i.e., the one for which you provide the absolute HTTP and FTP paths in your client registration). If you register your web site root as your WB projects folder, it will treat every folder thereunder as a WB project and scan whatever you locate there for inclusion.

Perhaps it would be easier to consider this WinBuilder DBMS application as if it were two related but separate applications: (1) the database itself and (2) the "tools" provided by the client package that can be downloaded and installed onto a client's WB projects site.

(1) There is only one database. That same unified database contains any and all of the data that gets entered into it by anyone from any location and it provides unified responses to all of the queries that are issued to it by anyone from any location. Any ordinary user (i.e., anyone who hasn't logged in) sees only the data that have been entered into that database (by someone somewhere) and that correspond to whatever SQL query the user interface issues to it on behalf of that user. Insofar as any such database responses are concerned, the location from which any query originates is totally irrelevant. When you update the database by entering your project(s) data into it, you are making that data available to "the world". Whether one is browsing/searching via the user interface on the demo site, on your client site, or on the dB host site makes no difference whatever to the query responses that are issued and received. You might even say that's the whole point.

(2) The "tools" that are contained in the client package that you download and install onto your WB projects site are something entirely different. When you log into that client package (and ONLY that client package) its physical location isn't merely important. It's absolutely critical at least insofar as the PHP tools are concerned. (The FTP tools work differently, of course.) The PHP tools must be able to translate the client registration info that you provide into the real paths to the WB project folder within your site on your server. They can only do that if they are also located somewhere within that site (i.e., somewhere that actually has a relative path to the projects folder). You simply cannot use any other package (whether client or host) to log in and use those PHP tools for manipulating the project files that are physically located on your site. On the other hand, so long as the client package is installed within your site and can thus `find` the WB project folders, it really doesn`t care what (sub)folder it is in itself. So, I guess you can install it into multiple folder locations if you really want to.

In summary, the user interface, which works exclusively with data, cannot be lumped together with the registered client interface, which works mainly with folders and files. They are two very different beasts. You can wander all over the map and get the same data query results with the user interface. You can NOT, however, expect to get the same results for physical file listings, etc., when you log in to the demo site (or any other) as you get when you log into your own client package on your own site. Neither can I or anyone else. That just ain`t gonna happen ... and, believe me, you wouldn`t want it to. You need to stop thinking about it as some kind of discrepancy. It`s not.

... with 'dot's in my project folder names, like 'pe21.001'. I hadn't clicked 'Update DB' or 'Create INI' before, but was trying to expose all files and folders on my server. The 'Update DB' generated a string of errors about directories (named without dots) not found.

To be honest, I`ve never actually tested it with (sub)folders that include dots in their names. I can`t think of any reason why it should have any problem with that any more than it does with file names that include dots, but I`ll have to dig into that one and get back to you. If it`s not handling that properly, you definitely have found an unacceptable problem. Thanks.

[allanf]

Arvy,

Eventually things might just 'click' in my mind. Your concerns about security are admirable.

I was looking at things from a non-logged-in user's point of view, and thinkng, "What is that WBDBClient folder doing there?" Here's a screenshot:



One of my packages is at the highest level from a ftp perspective:
'ftp://pe2.winbuilder.net/WBDBClient/'.

The other is at the highest level from a http perspective:
'http://pe2.winbuilder.net/WBDBClient/' which is equivalent to the ftp 'ftp://pe2.winbuilder.net/pe2.winbuilder.net/WBDBClient/'.

The latter is the accessible one, which I have called 'active' and log into.

My project directories are also at the highest level from a http perspective. For example:
'http://pe2.winbuilder.net/pe21.001/' or
'ftp://pe2.winbuilder.net/pe2.winbuilder.net/pe21.001/'

So, in the Client Domain Setup form, I have the absolute HTTP path set at '/' which causes the Client Package to expose itself to the WWW. Compare my setup with your Test Client setup.





I think that the best course would be to shift the projects down a level to say:
'http://pe2.winbuilder.net/projects/' or 'ftp://pe2.winbuilder.net/pe2.winbuilder.net/projects/'
and reset the Client Domain Setup, so that only the '/projects/' directory is exposed to queries from non-logged-in users.

Does that make any sense? It's barely comprehensible to me. ... ...

[Arvy]

Yes, I think it would make a lot of sense, especially if you want to be selective about including some things and not others. I'm working on a feature that would allow "selectivity" within a client's registered WB projects folder, but it's not quite ready for prime time yet.

If you think of your web site root as being something like your WinBuilder %basedir%, then you would normally put your WB projects into their own projects folder thereunder and subfolders subordinate to that. Having done that, you would only need to edit your client registration info to make that your registered main WB projects folder. And, if you wish, you could even have other project folders (e.g., WinBuilder beta) that would be outside the registered scope entirely. For that matter, you could organise your site with multiple "main" WB projects folders and register each separately (e.g., for "guest developers") but we'll avoid getting into any such needless complications in this case.

The location(s) of your client package is quite unimportant so long as (1) it's somewhere within your own site and (2) it's not located under the registered projects folder. Provided that those conditions are met, it will find the correct relative path to the main WB projects folder based on your registration entry and it will not include itself in the dB updates. You definitely do not need your client package to be installed twice unless you want it that way. A single client package installation can handle both the HTTP and FTP paths that you provide for accessing the project folder.

Hope that helps. I'll get on top of that folders-with-dots-in-their-names issue ASAP.

[allanf]

Thanks.

[Arvy]

The name-dot-extension folder handling problem in the client download package has been identified and corrected. For your convenience, a zipped copy of the revised WBDBClient/includes/wbfuncs.php file is attached to this post. Just extract, upload and overwrite the existing file of that name in the client package that you installed on your web site.

For those who may be curious, an "over-enthusiastic" string replacement in a path cleaning routine was getting rid of too many dots.

 wbfuncs_php_fix.zip   7.92KB   464 downloads

[allanf]

... an "over-enthusiastic" string replacement in a path cleaning routine was getting rid of too many dots.

rofl....