(ˢᵒᶜⁱᵉᵗʸlog)
(ˢᵒᶜⁱᵉᵗʸserver)
(ˢᵒᶜⁱᵉᵗʸtask)
Chandler
REST
angular.js
coding-for-fun
cordova
css
hackerspace
installation
javascript
mockups
multi-language
nReduce
pike
publishing
retreat
sTeam
stylesheets
user-interface
virtual hosting
xiamen

The REST API is developed as an application inside the sTeam server. This simplifies development quite a lot, as we don't need to restart the server for every change. Instead the just the API code gets updated and reloaded. It may eventually be integrated into the core, however the longterm plan is actually to move functionality out of the core, to make development easier.

Over the summer, Trilok has been improving the export and import tools, so now we can finally export the whole development history of the REST API to git, and we can import that git history into other servers. Using git, we can also easily merge development done on different servers.

To get the current version of the API clone the repo into your home or to any place where you keep your development repos. Then change to the tools directory of your installation and run import-to-git.

git clone https://github.com/societyserver/steam-rest
cd steam-rest
git checkout origin/rest.pike
export steamrest=`pwd`
cd /usr/local/lib/steam/tools
./import-from-git.pike -u root $steamrest /

It is important that the first import is done as root because the API code needs to run with root privileges and it will only do that if the object that holds the source is created as root.

Once the api code is loaded there are just a few tweaks needed to make it work.

We need to fix the mime-type, as the import script is not doing that yet.

OBJ("/sources/rest.pike")->set_attribute("DOC_MIME_TYPE", "source/pike");

changing the mimetype will change the class of the rest api script from Document to DocLpc.

> OBJ("/sources/rest.pike");                                               
(1) Result: 127.0.0.1:1900/rest.pike(#840,/classes/Document,17,source/pike)
> OBJ("/sources/rest.pike");                                               
(2) Result: 127.0.0.1:1900/rest.pike+(#840,/classes/DocLpc,529,source/pike,0 Instances, ({  }))

this takes a moment, check the type a few times until it's done.

Then instantiate an object from the source, give it a proper name, and move it to the /scripts/ container"

object rest = OBJ("/sources/rest.pike")->provide_instance();
rest->set_attribute("OBJ_NAME", "rest.pike");
rest->move(OBJ("/scripts/"));

Instantiating the object needs to be done as sTeam-root, in order for it to have permissions to run on behalf of other users.

Once this is done you are ready to start using the API.

Author :mbaehr   |   size :2603Bytes   |   Publish Date :Jan/14/16/08:18   |   to Top

After you installed and set up a basic sTeam server, you'll want to update to the latest development state. there are some significant changes in there, some of which are documented in the articles below.

Because we don't have a way to run a development server directly from the source repo, and we don't want to reinstall for every change, we instead track changes in a runtime repo directly from the directory where the server is installed. This will change eventually, but probably not until we have restored the original sTeam development history.

To get this runtime repo, you first clone https://github.com/societyserver/sTeam.git to a temporary directory:

cd /tmp
git clone https://github.com/societyserver/sTeam.git
cd sTeam

and check out the version that matches an initial installation:

git checkout 49a79885aa509427c818f8ecc6bf3c78e056b4be;

the worktree should now match your installation except for a few files. we can now copy the .git directory to our installation:

cp -avi .git /usr/local/lib/steam/;

and verify it:

cd /usr/local/lib/steam/;
git status
# HEAD detached at 49a7988
# Changes not staged for commit:
#   (use "git add/rm ..." to update what will be committed)
#   (use "git checkout -- ..." to discard changes in working directory)
#
#       deleted:    .gitignore
#       modified:   server/libraries/Test.pmod
#       modified:   version
#
# Untracked files:
#   (use "git add ..." to include in what will be committed)
#
#       bin/steam
#       server/libraries/coal.so
#       server/libraries/steamtools.so
#       server/libraries/wiki.so
#       server/libraries/xml.so
#       server/libraries/xslt.so
#       tmp/
no changes added to commit (use "git add" and/or "git commit -a")

you are missing .gitignore, so let's fix that:

git checkout -- .gitignore

run git status; again to see what that did:

# HEAD detached at 49a7988
# Changes not staged for commit:
#   (use "git add ..." to update what will be committed)
#   (use "git checkout -- ..." to discard changes in working directory)
#
#       modified:   server/libraries/Test.pmod
#       modified:   version
#
# Untracked files:
#   (use "git add ..." to include in what will be committed)
#
#       tmp/
no changes added to commit (use "git add" and/or "git commit -a")

most of the untracked files are gone, because they are listed in the .gitignore.

you may or may not see

#       modified:   server/libraries/Test.pmod
#       modified:   version

it depends on how you patched Test.pmod, and on your pike version:
git diff; will show you:

diff --git a/server/libraries/Test.pmod b/server/libraries/Test.pmod
index 4378768..aea5e3a 100644
--- a/server/libraries/Test.pmod
+++ b/server/libraries/Test.pmod
@@ -98,7 +98,7 @@ int start_test ( object|void obj ) {
   if ( suite["started"] == 0 )
     suite["started"] = time();
 
-  object first_test = TestCard( obj, obj->test, 0, 0);
+  object first_test = TestCard( obj, obj->test, 0, 0 );
   suite["pending_calls"] += ({ first_test });
   first_test->enqueue();
   return 1;
diff --git a/version b/version
index 7944427..b32c4b6 100755
--- a/version
+++ b/version
@@ -1 +1 @@
-Pike v7.8 release 700
+Pike v7.8 release 866

i have a newer pike version, and i added an extra space in the patch for Test.pmod

we can fix Test.pmod:

git checkout -- server/libraries/Test.pmod

we'll ignore version for now.

now that we are confident that the installation matches the repository, we can update to the latest development state:

git checkout origin/societyserver-devel -b `hostname`-devel;
git status;
# On branch -devel
# Changes not staged for commit:
#   (use "git add ..." to update what will be committed)
#   (use "git checkout -- ..." to discard changes in working directory)
#
#       modified:   version
#
# Untracked files:
#   (use "git add ..." to include in what will be committed)
#
#       tmp/
no changes added to commit (use "git add" and/or "git commit -a")

there! your server is now up-to-date. you can now track your changes, and send us patches. don't forget to restart.

and we can now use the updated tools to import the REST API.

Author :mbaehr   |   size :4610Bytes   |   Publish Date :Sep/5/15/08:08   |   to Top

these are the commands needed to install sTeam from source on CentOS 6.5

Pike

  yum install mysql-server mysql-devel bzip2-devel libjpeg-devel  libpng-devel zlib-devel pcre-devel freetype-devel gmp-devel make nettle-devel gcc tar
  # in centos 7 install mariadb-server instead
  get the pike source from http://pike.lysator.liu.se
  unpack...
  make
  make install

sTeam

  # get steam 2.9 from git repo
  git clone http://github.com/societyserver/sTeam.git
  cd sTeam
  git checkout steam-2.9-source
  # get webinterface
    https://raw.githubusercontent.com/societyserver/sTeam/steam-packages/web-2_2_17.spm
  # patch line 101 in server/libraries/Test.pmod
  # line should be "object first_test = TestCard( obj, obj->test, 0, 0);"
  yum install libxml2-devel libxslt-devel automake flex
  /etc/init.d/mysqld start
  # centos 7: systemctl enable mariadb; systemctl start mariadb
  # centos 7: patch xslt.c following these instructions
  ./build
  ./install
  # start the server:
  cd /usr/local/lib/steam/
  ./start

  # install the webinterface with the spm tool:
  /usr/local/lib/steam/bin/spm -i web-2_2_17.spm
  
  # login at https://localhost/ (username "root", password "steam")

  # optional: set up your system to start sTeam automatically
  chkconfig --add steam
  chkconfig steam on

at this point you should have a running server

for other linux distributions you may need to adapt the package names, and you may also run into versioning issues with regards to libxml and libxslt. in future versions i intend to remove the libxml/libxslt support. (pike itself has xml support for your development needs. libxml is only needed for libxslt, and i want to remove the latter.)

Author :mbaehr   |   size :2043Bytes   |   Publish Date :Aug/31/15/15:16   |   to Top

(ˢᵒᶜⁱᵉᵗʸserver) aims to be a platform for developing collaborative applications.

What are the elements of collaboration?

  • People
  • Documents
  • Communication

In a collaboration platform we need features to manage these elements.

One way to manage them is to organize and structure them.

People are organized in groups. Groups are structured into hierarchies. Documents are structured into topics, areas or containers. Communication is structured into topics, threads.

In this article we'll focus on documents and how to structure them.

The primary concept to structure documents in sTeam is a Room. Rooms can be nested, and they can have Doors or Gates to connect them. Doors can be one-way or bidirectional.

Imagine a library, each room is designated to a topic. The room is divided up by shelves, each about a different subtopic. There are doors that lead from one room to the next. Or visit an art-gallery. Again each room is designated to pictures of a certain topic. A guided tour lets you follow a path in one direction from one room to the next.

Now picture yourself as the librarian, or the gallery-curator. Not only will you want to read the books, or look at the pictures, but you are the one to decide on the structure, so you will want to layout the rooms and connect them according to the topics that you cover, and the order in which the gallery should be explored.

We need to find a way to model this kind of structure in a two-dimensional web-interface. The navigation between Rooms, the ability to create and move rooms, creating one-way or bidirectional Doors.

Now we need to look at the documents: There are many kinds of documents:

    markup-text (markdown, html, others)
  • xml, json, csv
  • source-code
  • image
  • audio
  • video
  • binary
  • object (instance of a script)

Some documents consist of different parts:

  • archive (zip, tar.gz)
  • multipart document
  • multilanguage document

We probably want to somehow distinguish those visually.

Again, you are the librarian or gallery-curator and you want to move books and pictures around to their topics. You will add new books, or update them with new editions. You will remove books that are obsolete, or move them into an attic. You may feature some books on a display table, and store others in the shelves.

You may arrange them by title, or by author, or maybe by date.

From this mental model we can extract a few actions that are common to all documents:

  • move
  • rename
  • copy
  • link
  • sort

You have a whole team of librarians, and each one is responsible for a specific area. Your librarians are working together to decide how the books should be structured. They add comments, descriptions, keywords and other attributes. They also decide who should be able to read a book. Is it suitable for children?

  • access control
  • annotations
  • keywords
  • attributes

Lets take a more concrete example: A group of academics working on a research project.

There is code, papers, reports, data, references to other work and to external resources. They will need to be organized in topics according to the different aspects of the research project. As the research progresses, and new things are learned, topics will need to be adjusted, papers rearranged, reports updated, references and connections between topics changed.

The structure lives and evolves. and so do the documents.

In conclusion, there is a lot of activity around document management, and there are many different scenarios possible.

Author :mbaehr   |   size :4081Bytes   |   Publish Date :Feb/7/15/05:49   |   to Top

sTeams way of handling multiple languages for a document currently works like this:

First, multi-language support needs to be configured by a sTeam administrator by specifying which languages are to be supported in the multilanguage_languages attribute on the admin group object. Once that is active, containers and rooms show an option to configure them as multi-language containers. If this is activated you are able to select a document in the container for each language that was configured above on the admin group.

This is rather suboptimal. Different pages may want to support a different set of languages. There is nothing in offering a certain set of languages that would require admin intervention. The above language list could be a hardcoded list covering all languages on the planet for example, but then we'd get a long list of languages to select in each container.

The next problem is that, in order to select a document for a language, the document needs to exist. So, first you need to create a document for a language, then go to the language setting for the container and select that document. And then better don't try to move the document into another container...

A better approach might be the following:

Throw away the admin setting. Just allow any container to be configured for multi-language use regardless. Then for each document or object even, specify its language, just like you currently specify the content-type. That language can then be reflected in the documents icon.

Optionally, if the container is a multi-language container then register the document with the container as the index document for that language.

With multiple languages, one document should be marked as the original. This document could then be used as a fallback if no suitable language is found. It would also allow marking other languages as up-to-date or not, as those should be newer.

Next, we need to decide if there should only be one document per language allowed, or multiple ones. The later might simplify some things, but complicate others, such as a translation interface which would need to differentiate which documents in each language are related. This question relates to the more general question of handling multi-part documents. A multi-part document would be a container with multiple documents inside which make up various parts, such as header, footer, sections, pictures, etc...

A multi-part document could be edited with all parts open in a separate text area at once, or all parts displayed and each to be made editable with a click. A multi-language document would show the original on top and all other languages below either with tabs or with an accordeon style interface opening one language at a time.

But how would we handle a multi-part and multi-language document? If the multiple parts are arranged from top to bottom, languages can be placed side-by-side perhaps? Put the original language on the left and select the target language on the right. In the inventory documents would be ordered by their appearance for each language.

How to implement all this?

  • Ignore the multilanguage_languages attribute and always allow a container to be configured as multi-language container.
  • Register a language attribute for the Document or even the Object class. (update: there already exists a OBJ_LANGUAGE attribute that appears suitable, it defaults to the language of the user)
  • Add a field to set the language in the object details.
  • Create an interface to edit a multi-language document. Given the above consideration this should probably be side-by-side even if it is only one document. This would also help in case the document is very long as you would be able to scroll and keep the translation always in view with the original.
  • In the admin view a multi-language container should have a direct edit-link to not require entering the container for editing.
  • When displaying a container with multiple articles, check if any containers inside are multi-language containers, and pick the appropriate content. Actually this can likely be done already in the xml generation stage, so that the xsl template does not need to deal with the multiple languages at all.
    • This can probably be done with a get_content() like function that calls get_content() on the appropriate language. For a multi-part document, this would have to concatenate the parts.
    • Ideally, a multi-language container would be able to look like a document in the xml structure. Can we do that? (update: if there is only one object per language we can probably handle this in get_public_inventory() by just returning the appropriate document instead of a container.
  • For a log, order the articles by container-creation time
Author :mbaehr   |   size :5143Bytes   |   Publish Date :May/13/12/14:44   |   to Top

(ˢᵒᶜⁱᵉᵗʸlog) is a simple stylesheet to create a weblog like presentation of sTeam content.
It is based on a heavely modified version of the internal public stylesheet and includes some helper functions from uni-pb style package.
The stylesheets are created by Chen Shi Jun and packaged by Martin Bähr

Features/Limitations

  • It currently handles only simple text files and does not allow adding comments.
  • Entries are displayed using the description as a title.
  • The menu is created from the Areas/Rooms
  • A Room named Topics is shown as a tag cloud with each Room inside listed as a topic in a size depending on the number of articles in the topic.

TODO list

  • Minor brush-ups on the style
  • Add a function to easely create new entries with links to topics.
  • Add support for Gates in the menu.
  • Support for comments
Author :mbaehr   |   size :1104Bytes   |   Publish Date :Dec/11/11/17:15   |   to Top

(ˢᵒᶜⁱᵉᵗʸlog) has been updated with various improvements and fixes:

  • Forms to create, upload and edit articles
  • Tabs to switch to the public view
  • Login and rss links
  • In the internal(admin) view, entries are displayed with description and topics.

The current version depends on the publishing and vhosting changes described in the previous post but future versions will hopefully support the old style as well

Author :mbaehr   |   size :553Bytes   |   Publish Date :Dec/26/11/18:38   |   to Top

We have redesigned the way objects are published and how virtual hosting is handled:

Publishing:

What was:

Each object used to have a Publication path that would set the absolute path for this object and all objects inside. This allowed a very flexible layout but could also lead to mixups and confusion if it wasn't used consistently.

More importantly it allowed one user to override the publication of another: if you published something under /hello/ i could publish something else as /hello/world/ and thus block an object called world that you might have had inside of /hello/

What is:

This Publication path is no longer used.

Instead, each group or user is allowed to set one area as their public area in the attribute GROUP_PUBLICROOM or USER_PUBLICROOM. That area is then published under the group or username. If the attribute is not set then it defaults to GROUP_WORKROOM and USER_WORKROOM respectively. Thus publishing something as /hello/ can only be done by a group or user named hello. And publishing something as /hello/world/ can only be done by someone who has the permission to write into the container published as /hello/.
Btw: Users and groups are now no longer accessed on /home/ but directly on /.

What may be:

The remaining work mostly centers around making publising more visible. It should be easier to see if an object is published, and where. And also switch publishing on and off. First steps of that are already visible in the (logⁿ) package which sports a tab for each virtual host where a container is published.

Virtual hosting

What was:

Virtual hosting used to be implemented very simple by setting a path that was just prepended to the actual path for each request. The request was actually rewritten and it was not recognizable that a virtual host was effective.

What is:

Virtual hosting has been redesigned completely:
A hostname is now assigned to a group or user.
When it is resolved, the group or user object is checked for a public room and the path of the request is then found inside it. If it is not found there, it is checked in the root-room and if that fails too, the first component of the path is checked to see if the group contains a subgroup or has a member by that name.

Because virtual hosting closely relates to publishing, the code to handle this has been made part of the filepath:url module. The functions to resolve a path to an object have been rewritten to handle the new way of publishing and virtual hosting. object to path translation also takes virtual hosting into account.

To handle the latter the Caller module was extended by a function to find the socket from which a request was made. Similar to CALLER, the new function Caller.get_socket() will traverse the backtrace until a socket object is found. The socket is then queried for the virtual host that is to be applied for the translation.

What may be:

Ideas range from automatically setting a subdomain for a subgroup or member, so that each subgroup or member can have its own hostname based on the group or username, to allowing a group to set their own domain.

Author :mbaehr   |   size :3608Bytes   |   Publish Date :Dec/24/11/18:12   |   to Top

One of sTeams features is link objects. That is objects that point to other objects. (ˢᵒᶜⁱᵉᵗʸlog) uses link objects to build topical logs. Each topic links to articles related to that topic.

sTeam link objects are not transparent. When you access a link object it needs to be handled specifically to extract the object (in our case the article) it points to.

Handling link objects separately has the advantage however that the target article can be adapted to the location of the link object.

One case where this matters is if an article contains relative urls. If the article /a/hello includes an image src="logo.png", then the browser will load the image from /a/logo.png. Now if /b/world is a link object for /a/hello it would try to include the image from /b/logo.png. Obviously, that won't work, so when presenting the article though the link object we need to parse it and replace relative urls so that they all point to items in relation to the location of /a/hello

The rss feed has also been improved to handle link objects, but in difference to above, here the link object is simply transparently replaced with the original object. Because the feed is accessed in a different context from the whole site, the link location is less relevant. Also if an article is linked from multiple feeds, all feeds will show the same object, thus allowing the feedreader to notice duplicates.

Author :mbaehr   |   size :1531Bytes   |   Publish Date :Jan/10/12/17:44   |   to Top