A New Linux Distro for 2016 (Compiling...)

I'm getting back into more technical projects than the many research topics I have been working on of late and I am going to turn this blog into a publishing ground for these projects. A streamlined version of what is here will probably end up on my Github for those who are interested.

The first project is to do another Linux setup with one of the more complicated distros: Gentoo. I had machines running earlier versions of Ubuntu (over 5 years ago) and CrunchBang (2 years ago). In what follows, I will discuss some of the challenges I ran into during the initial installation. Subsequent posts will discuss other details.

System Specs:

Acer V3-571-6447 (refurbished)

Intel Core i3-2328M

Intel HD Graphics 3000

Atheros 9k Wifi Card

I mainly followed the handbook for amd64. If you are using something else, the approach might be different.

Using Gentoo amd64 minimal installation CD (date stamped 2016-02-18).

I burned it to a DVD ROM rather than a CD. No problem.

Challenge 1: UEFI

The system was set up with native Windows 8 support so you can't boot to the UEFI menu from startup. I found out how to get there from this site.

Following these instructions, I then turned off UEFI, rebooted, hit F2 for bios, and then swapped the boot order so the disc drive was highest priority.
Easy enough.

Challenge 2: Wifi Setup

Setup for Atheros 9k Card driver went well (modprobe ath9k).

My network was running WPA encryption so I had to switch to WEP-ASCII in order to get the network working properly. I intend to switch it back once I have the system up.
ifconfig gave me the network names and net-setup <wifi name> provided the initial setup with dhcpcd <wifi name> providing the means to get the network off the ground (i.e., successful ping of google.com).

Challenge 3: Wifi Hell
After I booted into the new kernel, I found that I didn't have access to iwconfig, so I had to boot back into the livecd and emerge --ask net-wireless/wireless-tools. My suggestion is that you do this before this point.

After that was fixed, I couldn't seem to get the wifi up. But ,there seems to be an extremely strict order to the setup. First you modprobe ath9k then you take down the network with ifconfig <wifi name> down. The next step is to set the ESSID and key with iwconfig as follows:
iwconfig <wifi name> essid "<name of network>"
iwconfig <wifi name> key s:"<ASCII-password>"
After that, ifconfig <wifi name> up has to be called before dhcpcd <wifi name> is called. That got the wifi up and running for me when no other combination or order could.

Challenge 4: Adding wpa_supplicant
This was surprisingly easy. Just follow the instructions here. Honestly, I would suggest emerging this to the new kernel from the livecd rather than dealing with iwconfig.

Note: you have to /etc/init.d/dhcpcd restart before trying to run anything. A fresh reboot always works, too.

(as one line)
wpa_supplicant -B -i<wifi name> 
-c/etc/wpa_supplicant/wpa_supplicant.conf
is also a good command to get things rolling.

Challenge 5: Xorg
Turns out you can modify the kernel directly from within the kernel easily enough as shown here. I followed instructions for Xorg's kernel setup from here. I also built-in synaptic's touchpad source files as follows:

Device Drivers --->
  I2C support --->
    -*- I2C support
  Input device support --->
    Mice --->
      <*> Synaptics I2C Touchpad support
      <*> Synaptics USB device support

Challenge 6: systemd Nightmare
I got it up and running but I am not even confident what I did. I used these three websites (1, 2, 3).

Steps that I am confident happened:

Update USE:
nano -w /etc/portage/make.conf
add systemd dbus -consolekit to USE=

Using genkernel-next:
emerge -aC sys-kernel/genkernel
emerge -a sys-kernel/genkernel-next

To begin source set and installation (I did not use lvm or udev for initramfs though both sources were available in the kernel):
genkernel --menuconfig --install all

This source parameter was set:
Gentoo Linux --->
  Support for init systems, system and service managers --->
    [*] systemd

Symlink for mtab:
ln -sf /proc/self/mounts /etc/mtab

Install systemd (I tried setting the USE before installing and it failed):
emerge -av systemd

Updated the system:
emerge -avuND @world

Update grub (real_init worked in the final attempt... did not care to try if init would work):
GRUB_CMDLINE_LINUX="real_init=/usr/lib/systemd/systemd"

And rebooted.

Challenge 7: NetworkManager
Had to recompile the source for a couple options listed here. In emerge'd dhclient first then set dhclient and networkmanager into the USE of make.conf. Then I emerge'd networkmanager.
emerge -a net-misc/dhcp
emerge -a net-misc/networkmanager

I then shut off dhcpcd with:
/etc/init.d/dhcpcd stop

Then booted up NetworkManager through systemd:
systemctl start NetworkManager

This initialized the network but it needed to be connected. I looked up dhclient and found the command line call here.
nmcli dev wifi connect <ssid> password <password>

This got me up and running. But, as a final cleanup I removed dhcpcd and enabled NetworkManager on boot.
emerge -cav dhcpcd
systemctl enable NetworkManager

Challenge 8: Gnome
I just followed the instructions for gnome-base/gnome-light, which can be found here.

Cultural Engineering and Expression: Embassytown

Abstract:

A brief discussion of the fascinating implications of the linguistic architecture of China Miéville's Ariekei of Embassytown. This largely focuses on the deliberate cultural engineering of linguistic forms and expression. There are spoilers in this discussion, but none related to the plot in any direct way.

I recently began reading (or listening to; I have a new found adoration for audio books) Embassytown by China Miéville and, besides being absolutely fantastic conceptually, it has really captured my imagination of a possible future. In the story there is a species, the Ariekei, that can only understand communicatory acts if they are first instantiated in a signifying event. This means that, if I wanted to say "it hit my nerves like the grating wail of a thousand pounds of tinfoil in self-adjacent laceration," I would actually have to go get a thousand pounds of tinfoil and have it effect the collective perception as described. I would then refer back to that moment in order to make the expression intelligible.

What is interesting is that humans are actually like this. If the referent is lacking, the statement is lost; though, we can generalize to some degree. This is why my simile is probably comprehensible despite the fact that few people would ever have even seen a thousand pounds of tinfoil. However, in highly theoretical, hyper-contextual, or inordinately abstract domains, this problem resurfaces. People lack the eventful referent that
places the relevant topic in a space of intelligibility. As a consequence, when I am discussing these topics, I often take a shotgun approach with various metaphors, similes, and the like until one 'catches' in the listener. I then use that hook to build an architecture of comprehension around the topic in question.

The Ariekei do something similar but as a culture and species. Thus, in some way (as far as I am in the text, this is not yet clear) they sense that a certain form of expression is needed and they create the conditions for its manifestation. This then allows that expression to be made in future, applicable situations. I think this is a brilliant idea.

Imagine an international organization, something like NATO, whose sole function is to be sensitive to global fluctuations in cultural presence (local instantiations are equally viable and perhaps more desirable, depending on one's political proclivities). When the conditions suggest a need for a new form of expression, they collect all the elements (and, of course, it would be an honour to be so collected) and generate an event for global consumption. The entire world, as a whole, now can mutually recognize this experience and use it to communicate the underlying network of relations that were instantiated in that event. An easy (if not cutting edge) example that would be very relevant and useful today would be a global communicatory event for something like "unity" or "friend."

Religion, philosophy, and mysticism of various kinds has often operated as what I would call an unreflexive form of this organization. They manufacture clusters of these metaphoric referents that people use to grasp otherwise unintelligible experiences, but they do so in a way that is not aware of what it is doing. Usually this is because, in all but the most 'advanced' forms, the proponents of these views are entirely unaware that they are engineering as opposed to discovering (or that discovery is actually a creative act). Marketing and propaganda, on the other hand, self-consciously manufacture culture, but they do so as a byproduct. Science is somewhere in between.

As with many things I say, many might argue that this phenomenon already exists in a non-organizational form distributed within culture; thus, it is a given and mostly uninteresting. I completely agree, at least with the former. The issue is that this is equally unreflexive and, consequentially, it is driven almost entirely by chance and herd mentality. To build an international organization and navigate all the ethical, social, political, and legal shoals that entails would motivate an evolution in human thinking that is sorely needed. That itself would be an event worth communicating, an expression of yet unheard signification.

Images courtesy of:
http://vincentchongart.wordpress.com/2011/05/24/embassytown-finished-cover-design/
http://www.wallpapervortex.com/digital_art-abstract-wallpapers.html#.Uq8jbfRDt8E
http://euobserver.com/defence/27860
http://johnnyholland.org/2009/09/good-ixders-borrow-great-ones-steal/

Setting up matplotlib Python Package on a Mac

Thanks to this post, I've managed to get python and a few packages working on a Hackintosh running Snow Leopard. However, I found matplotlib to be particularly challenging, even with the instructions. Hence, for myself and others, I have decided to record my insights.

Once you have homebrew, python, and numpy installed as indicated in the post, above, do as follows in a terminal:

brew install freetype
brew install libpng

chmod +w /usr/local/lib
brew link freetype --force --overwrite

The last line should fail, but list a couple of files. Find the files (it tells you where they are) and delete them (or move them into a new folder for safe keeping). Retype the command and the link should occur. Then:

chmod +w /usr/local/include
brew link libpng --force --overwrite

As in the previous case, this will fail and list a few files. Proceed as in the previous case. Retype the command.

Finally type:

pip install matplotlib
or
pip install git+git://github.com/matplotlib/matplotlib.git#egg=matplotlib-dev

You'll have to use the latter if you're using the Mountain Lion OS. It was the latter that I concluded with, personally.

Disjoint-set Data Structure in Python

I needed this code to write Kruskal's minimum spanning tree algorithm.

class DisjointDataStruct(object): 
  """Disjoint-set data structure. 

  For details see:
  http://en.wikipedia.org/wiki/Disjoint-set_data_structure

  Public functions:
  union(x, y)
  find(x)
  """

  def __init__(self, vertices):
    """My personal representation of the spaghetti stack.

    vertices - a list of vertex names from a graph
    """
    self.struct = {v: [v, 0] for v in vertices}

  def union(self, x, y):
    """Joins subsets x and y into a single subset.

    Uses union by rank optimization.

    x, y - vertex names corresponding to keys in self.struct
    """
    xRoot = self.find(x)
    yRoot = self.find(y)
    if xRoot == yRoot:
      return

    xV = self.struct[xRoot]
    yV = self.struct[yRoot]
    if xV[1] < yV[1]:
      xV[0] = yRoot
    elif xV[1] > yV[1]:
      yV[0] = xRoot
    else:
      yV[0] = xRoot
      xV[1] += 1

  def find(self, x):
    """Determines which subset a particular element x is in.

    Uses path compression optimization.

    x - vertex name corresponding to key in self.struct
    """
    if self.struct[x][0] != x:
      self.struct[x][0] = self.find(self.struct[x][0])
    return self.struct[x][0]

The timings are as follows:
__init__ is 17.5ms per loop in 100 loops with 100000 vertices (using timeit)
union has a mean of 1.0ms, standard deviation of 1.10e-7, min of ~0.9ms, and max of 1.0ms.
The union data was computed over the 224 union time values that did not equal 0 over a total of 100000 random unions of 100000 vertices (using time.time end - start).

On a single run of cProfile with a vertex set of 100000, and 100000 random unions where the two elements are not equal (I increment one of them if they are), I get the following values:
0.042s for __init__
0.197s for union in total (100000 calls) so per call is ~1.97e-06s
0.130s for find in total (299990 total / 200000 primitive) so per call is ~4.33e-07s

All of this was clocked running an i5-2500K @ 3.3GHz

2D Wraparound World

I figured I would share a finding I had while working on a project. For those of you who have attempted to code a representation of a wraparound map (2D torus) using a 1 or 2D array, you may have found the experience quite challenging to get operating smoothly. I was trying to find a more elegant solution when I stumbled upon the following formula:


sqrt(min(|x1 - x2|, w - |x1 - x2|)^2 + min(|y1 - y2|, h - |y1-y2|)^2)

Where the two points are (x1, y1) and (x2, y2);
w and h are the width and height;
min and sqrt are minimization and square root functions, respectively.

With that the process is absurdly easy.

Note: If you just need relative distance for a comparison, don't bother using the square root.

Function courtesy of:
http://stackoverflow.com/questions/2123947/calculate-distance-between-two-x-y-coordinates/2123977#2123977

Python Nested List

In a previous post, I outlined a python function for creating nested lists. In retrospect, it was a naive, if functional, first attempt. As the size of the various dimensions increases, the function would become quite slow largely as a result of the deepcopy calls and recursion. A more efficient method would use the following syntax:

nestedList = [[[baseValue for x in range(axisLength)] 
                for y in range(axisLength)] for z in ...]

Now, a naive attempt to turn this into a function might do something like the following:

def nestedList(axisLength, degree, base):
  for x in range(degree):
    base = [base for y in range(axisLength)]
  return base

But this will actually just create shallow copies. The problem is that base is not recreated at each iteration. However, this is fixable with generators. For example:

def baseGen(base, axisLength):   
  while True:
    try: 
      yield [base.next() for x in range(axisLength)]
    except AttributeError:
      yield [base for x in range(axisLength)]

def nestedList(base, axisLength, degree):
  for d in range(degree):
    base = baseGen(base, axisLength)
  return base

Then, you would just call nList = nestedList(None, 5, 3) and then ls = nList.next() for a 5x5x5 cube with None as the base value. An added advantage of using generators is that you can just keep calling nList.next() every time you want a new cube.

Ingress: Part Game | Part Reality

Pseudo-Abstract:

A new Google game, Ingress, is discussed along with some of its implications for the narrowing divide between various domains of human experience.

Key Links:

Rachel Metz's post on Ingress.

A recent post by Rachel Metz of MIT Technology Review discusses a fascinating new Google app: Ingress. Ingress puts the player in an epic battle for a newly discovered resource, exotic matter, which is distributed throughout the world. Naturally, the nodes that spill this matter into the world, key tactical positions for either team in the game, are conveniently placed on landmarks and other useful locations. Thus, by playing the game, people flood Google with an endless wealth of location-based information. Genius.

By integrating game and application, Google successfully takes an excellent step into the realm of augmented reality. As I have mentioned previously (1, 2), this domain is ripe for exploration and Ingress is certainly at the forefront of this work.

What is central to this exploration is a fusion in the divide between domains we normally consider separate like game and application. This fusion is possible because the divide, though once very real, is becoming increasingly illusory. The human-machine distinction is becoming all but absurd. 'Real life' is so artificial that most people 'play' themselves while living in fictional realities that they find more meaningful. The result is a convergence of sorts towards something like a singularity: a world in which humanity 'plays' the game of life, the good and the bad, purely for the sake of enjoyment and the base needs of all are met merely as a byproduct of this game.

Pictures courtesy of:

http://www.t3.com/news/google-ingress-app-uses-augmented-reality

http://www.district78.com/tag/game-on/